Fastest way to convert big raster to polyline using R or Python?Automatic point plotting in ArcGIS from...

How to not let the Identify spell spoil everything?

Lightning Data Table inline edit

Does the US government have any planning in place to ensure there's no shortages of food, fuel, steel and other commodities?

Plausible reason to leave the Solar System?

Which RAF squadrons and aircraft types took part in the bombing of Berlin on the 25th of August 1940?

Charging phone battery with a lower voltage, coming from a bike charger?

What to do with threats of blacklisting?

Potential client has a problematic employee I can't work with

Crack the bank account's password!

Will rerolling initiative each round stop meta-gaming about initiative?

If angels and devils are the same species, why would their mortal offspring appear physically different?

What is a good reason for every spaceship to carry a weapon on board?

Converting very wide logos to square formats

Cat is tipping over bed-side lamps during the night

Not a Long-Winded Riddle

Time-efficient matrix elements grouping and summing

Need help with a circuit diagram where the motor does not seem to have any connection to ground. Error with diagram? Or am i missing something?

Why avoid shared user accounts?

The No-Straight Maze

Can my friend and I spend the summer in Canada (6 weeks) at 16 years old without an adult?

Why did Luke use his left hand to shoot?

Taking headphones when quitting job

How can I prevent an oracle who can see into the past from knowing everything that has happened?

Subsurf on a crown. How can I smooth some edges and keep others sharp?



Fastest way to convert big raster to polyline using R or Python?


Automatic point plotting in ArcGIS from thresholded imageHow to get X Y coordinates and cell value of each pixel in a raster using Python?How to get raster corner coordinates using Python GDAL bindings?Convert a classification image to aggregated polygon in shapefile using gdal/pythonHow to find the distance between start of line and start of raster with arcpy/python?How to improve performance when using a large amount of data (big tables) in ArcGIS?Extract raster values to multipoint shapefile and write to new field using gdal/pythonHow to speed up Select by location?Programmatic Zonal statistics to obtain cell value at 2% and 98% cumulative frequency IN GDAL OR PYTHONvectorize raster image containing linesAligning many rasters using PyQGIS or Python?













11















I have a big raster file (129600 by 64800 pixel) with global water bodies (1 bit values 0 and 1) and try to extract ocean and inland water shorelines.



I've tried with ArcGIS and QGIS to convert from raster to polyline, but it takes ages.



Does anybody know a better/faster way (Python or R) or a better tool for this task?



Update




  • R: rasterToContour might be fast and precise but if you have a very large dataset like mine (8,398,080,000 pixels) you need either a very big amount of RAM (more than 16GB) or you force R to do more processing on the hard drive and it will also take ages.

  • Python/GDAL: gdal_poligonize creates polygons instead of polylines


Update 2




  • R rasterToContour: rasterToContour does not deliver the wanted results. Compared to ArcGIS (raster to polygon followed by feature to line) it does not extract the exact pixel outline, as shown in the examples below.


rasterToContour result
rasterToContour result



ArcGIS result
ArcGIS result



UPDATE 3



Python/GDAL: I've run gdal_polygonize from command line against ArcGIS on a test dataset and the results were extremely clear:




  • gdal: 49 seconds

  • ArcGIS: 1.84 seconds










share|improve this question

























  • Did that, see Update 3.

    – Generic Wevers
    Oct 16 '15 at 13:06











  • Can you provide that test dataset, so we can see if proposed alternatives are faster and/or produce the required results?

    – Kersten
    Oct 17 '15 at 11:35
















11















I have a big raster file (129600 by 64800 pixel) with global water bodies (1 bit values 0 and 1) and try to extract ocean and inland water shorelines.



I've tried with ArcGIS and QGIS to convert from raster to polyline, but it takes ages.



Does anybody know a better/faster way (Python or R) or a better tool for this task?



Update




  • R: rasterToContour might be fast and precise but if you have a very large dataset like mine (8,398,080,000 pixels) you need either a very big amount of RAM (more than 16GB) or you force R to do more processing on the hard drive and it will also take ages.

  • Python/GDAL: gdal_poligonize creates polygons instead of polylines


Update 2




  • R rasterToContour: rasterToContour does not deliver the wanted results. Compared to ArcGIS (raster to polygon followed by feature to line) it does not extract the exact pixel outline, as shown in the examples below.


rasterToContour result
rasterToContour result



ArcGIS result
ArcGIS result



UPDATE 3



Python/GDAL: I've run gdal_polygonize from command line against ArcGIS on a test dataset and the results were extremely clear:




  • gdal: 49 seconds

  • ArcGIS: 1.84 seconds










share|improve this question

























  • Did that, see Update 3.

    – Generic Wevers
    Oct 16 '15 at 13:06











  • Can you provide that test dataset, so we can see if proposed alternatives are faster and/or produce the required results?

    – Kersten
    Oct 17 '15 at 11:35














11












11








11


2






I have a big raster file (129600 by 64800 pixel) with global water bodies (1 bit values 0 and 1) and try to extract ocean and inland water shorelines.



I've tried with ArcGIS and QGIS to convert from raster to polyline, but it takes ages.



Does anybody know a better/faster way (Python or R) or a better tool for this task?



Update




  • R: rasterToContour might be fast and precise but if you have a very large dataset like mine (8,398,080,000 pixels) you need either a very big amount of RAM (more than 16GB) or you force R to do more processing on the hard drive and it will also take ages.

  • Python/GDAL: gdal_poligonize creates polygons instead of polylines


Update 2




  • R rasterToContour: rasterToContour does not deliver the wanted results. Compared to ArcGIS (raster to polygon followed by feature to line) it does not extract the exact pixel outline, as shown in the examples below.


rasterToContour result
rasterToContour result



ArcGIS result
ArcGIS result



UPDATE 3



Python/GDAL: I've run gdal_polygonize from command line against ArcGIS on a test dataset and the results were extremely clear:




  • gdal: 49 seconds

  • ArcGIS: 1.84 seconds










share|improve this question
















I have a big raster file (129600 by 64800 pixel) with global water bodies (1 bit values 0 and 1) and try to extract ocean and inland water shorelines.



I've tried with ArcGIS and QGIS to convert from raster to polyline, but it takes ages.



Does anybody know a better/faster way (Python or R) or a better tool for this task?



Update




  • R: rasterToContour might be fast and precise but if you have a very large dataset like mine (8,398,080,000 pixels) you need either a very big amount of RAM (more than 16GB) or you force R to do more processing on the hard drive and it will also take ages.

  • Python/GDAL: gdal_poligonize creates polygons instead of polylines


Update 2




  • R rasterToContour: rasterToContour does not deliver the wanted results. Compared to ArcGIS (raster to polygon followed by feature to line) it does not extract the exact pixel outline, as shown in the examples below.


rasterToContour result
rasterToContour result



ArcGIS result
ArcGIS result



UPDATE 3



Python/GDAL: I've run gdal_polygonize from command line against ArcGIS on a test dataset and the results were extremely clear:




  • gdal: 49 seconds

  • ArcGIS: 1.84 seconds







python r performance






share|improve this question















share|improve this question













share|improve this question




share|improve this question








edited Oct 17 '15 at 10:50







Generic Wevers

















asked Oct 16 '15 at 8:56









Generic WeversGeneric Wevers

620413




620413













  • Did that, see Update 3.

    – Generic Wevers
    Oct 16 '15 at 13:06











  • Can you provide that test dataset, so we can see if proposed alternatives are faster and/or produce the required results?

    – Kersten
    Oct 17 '15 at 11:35



















  • Did that, see Update 3.

    – Generic Wevers
    Oct 16 '15 at 13:06











  • Can you provide that test dataset, so we can see if proposed alternatives are faster and/or produce the required results?

    – Kersten
    Oct 17 '15 at 11:35

















Did that, see Update 3.

– Generic Wevers
Oct 16 '15 at 13:06





Did that, see Update 3.

– Generic Wevers
Oct 16 '15 at 13:06













Can you provide that test dataset, so we can see if proposed alternatives are faster and/or produce the required results?

– Kersten
Oct 17 '15 at 11:35





Can you provide that test dataset, so we can see if proposed alternatives are faster and/or produce the required results?

– Kersten
Oct 17 '15 at 11:35










4 Answers
4






active

oldest

votes


















4














I'm working with R and used rasterToPolygons from the raster package in the past, but now I prefer gdal_polygonizeR by John Baumgartner. It bases on gdal_polygonize.py and is much faster.
John Baumgartner published the code and gave an example for usage in his blog.



If you are familiar with python you could use gdal_polygonize.py directly of course.






share|improve this answer



















  • 1





    I'll give it I try. Last time I've used gdal_polygonize.py ArcGIS was still faster.

    – Generic Wevers
    Oct 16 '15 at 10:12











  • I didn't expect that ArcGis can be faster that gdal. @Generic Militzer

    – Iris
    Oct 16 '15 at 10:26











  • Ah wait, this will create polygons but I need polylines.

    – Generic Wevers
    Oct 16 '15 at 10:28











  • If you put your data into a File Geodatabase it is pretty fast. But still not fast enough. That's why I'm searching for alternatives.

    – Generic Wevers
    Oct 16 '15 at 10:30






  • 2





    It's not necessarily a problem that you get polygons, you can always convert them to polylines (although, with that many it could of course take a while as well).

    – Martin
    Oct 16 '15 at 11:40



















4














Try rasterToContour from the raster package.



f <- system.file("external/test.grd", package="raster")
r <- raster(f)
r[r[] < 750] <- 0
r[r[] >= 750] <- 1

x <- rasterToContour(r)
class(x)
> [1] "SpatialLinesDataFrame"
> attr(,"package")
> [1] "sp"

plot(r)
plot(x, add=TRUE)


enter image description here



You may then easily write the files to a local folder, e.g. as 'ESRI Shapefile' (.shp), using the below code. Have a look at ogrDrivers from rgdal to find out which drivers your system is compatible with.



library(rgdal)
writeOGR(x, dsn = getwd(), layer = "coastlines", driver = "ESRI Shapefile")





share|improve this answer


























  • I'll try and keep fingers crossed it will not kill my RAM. Even though I have 16GB, which hopefully is enough, R sometimes is not so efficient with big raster files. But let's see.

    – Generic Wevers
    Oct 16 '15 at 9:06











  • Conversion worked somehow, but I wasn't able to check in detail. As I am usually more into raster data processing, can you tell me how I can transfer the SpatialLineDataFrame into a shapefile or something comparable. I've googled and still struggling, as I don't know the layer name (OGRwrite).

    – Generic Wevers
    Oct 16 '15 at 10:10











  • Haha, I definitely see your point. See above update.

    – fdetsch
    Oct 16 '15 at 10:25








  • 2





    Another hint: try to set 'maxpixels' in rasterToContour to some higher value, e.g. 1e+9. You'll end up with more details then. The default setting creates quite generalized contour lines.

    – fdetsch
    Oct 16 '15 at 10:46








  • 1





    If you're not willing to resample your data to a coarser spatial resolution, the only solution I can imagine then would be to split your data into multiple tiles (e.g. 16 sub-rasters), then perform rasterToContour on each tile separately in an iterative manner and, finally, merge the resulting shapefiles into one huge shapefile. In case you are interested, our working group's package Rsenal offers a function called splitRaster to create multiple sub-rasters from one huge raster.

    – fdetsch
    Oct 16 '15 at 11:42





















2














While I'm a big fan of GDAL, the polygonize tool was way too slow for my applications as well.



A fast alternative is gdal_trace_outline from Dans GDAL scripts which also has more options regarding tolerance, donuts, etc.



Like gdal_polygonize this also produces polygons which you'd need to convert afterwards with ogr2ogr -nlt MULTILINESTRING.



Downside to that is you need to compile it yourself, unless you are on a Linux or Mac OsX System.






share|improve this answer
























  • Unfortunately it failed with the error message: "Segmentation fault (core dumped)". I am guessing my file is too big or more precise it will produce too many small polygons.

    – Generic Wevers
    Oct 19 '15 at 11:35



















0














For posterity, I've been having success with the stars:: package in R for doing this type of operation quickly.



library(raster)
library(stars)
library(sf)
library(magrittr)

f <- system.file("external/test.grd", package="raster")
r <- raster(f)
r[r[] < 750] <- 0
r[r[] >= 750] <- 1

x <- st_as_stars(r) %>%
st_as_sf() %>% # this is the raster to polygons part
st_cast("MULTILINESTRING") # cast the polygons to polylines

plot(x)


enter image description here



plot(r)
plot(x, add = TRUE)


enter image description here





share























    Your Answer








    StackExchange.ready(function() {
    var channelOptions = {
    tags: "".split(" "),
    id: "79"
    };
    initTagRenderer("".split(" "), "".split(" "), channelOptions);

    StackExchange.using("externalEditor", function() {
    // Have to fire editor after snippets, if snippets enabled
    if (StackExchange.settings.snippets.snippetsEnabled) {
    StackExchange.using("snippets", function() {
    createEditor();
    });
    }
    else {
    createEditor();
    }
    });

    function createEditor() {
    StackExchange.prepareEditor({
    heartbeatType: 'answer',
    autoActivateHeartbeat: false,
    convertImagesToLinks: false,
    noModals: true,
    showLowRepImageUploadWarning: true,
    reputationToPostImages: null,
    bindNavPrevention: true,
    postfix: "",
    imageUploader: {
    brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
    contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
    allowUrls: true
    },
    onDemand: true,
    discardSelector: ".discard-answer"
    ,immediatelyShowMarkdownHelp:true
    });


    }
    });














    draft saved

    draft discarded


















    StackExchange.ready(
    function () {
    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fgis.stackexchange.com%2fquestions%2f166753%2ffastest-way-to-convert-big-raster-to-polyline-using-r-or-python%23new-answer', 'question_page');
    }
    );

    Post as a guest















    Required, but never shown

























    4 Answers
    4






    active

    oldest

    votes








    4 Answers
    4






    active

    oldest

    votes









    active

    oldest

    votes






    active

    oldest

    votes









    4














    I'm working with R and used rasterToPolygons from the raster package in the past, but now I prefer gdal_polygonizeR by John Baumgartner. It bases on gdal_polygonize.py and is much faster.
    John Baumgartner published the code and gave an example for usage in his blog.



    If you are familiar with python you could use gdal_polygonize.py directly of course.






    share|improve this answer



















    • 1





      I'll give it I try. Last time I've used gdal_polygonize.py ArcGIS was still faster.

      – Generic Wevers
      Oct 16 '15 at 10:12











    • I didn't expect that ArcGis can be faster that gdal. @Generic Militzer

      – Iris
      Oct 16 '15 at 10:26











    • Ah wait, this will create polygons but I need polylines.

      – Generic Wevers
      Oct 16 '15 at 10:28











    • If you put your data into a File Geodatabase it is pretty fast. But still not fast enough. That's why I'm searching for alternatives.

      – Generic Wevers
      Oct 16 '15 at 10:30






    • 2





      It's not necessarily a problem that you get polygons, you can always convert them to polylines (although, with that many it could of course take a while as well).

      – Martin
      Oct 16 '15 at 11:40
















    4














    I'm working with R and used rasterToPolygons from the raster package in the past, but now I prefer gdal_polygonizeR by John Baumgartner. It bases on gdal_polygonize.py and is much faster.
    John Baumgartner published the code and gave an example for usage in his blog.



    If you are familiar with python you could use gdal_polygonize.py directly of course.






    share|improve this answer



















    • 1





      I'll give it I try. Last time I've used gdal_polygonize.py ArcGIS was still faster.

      – Generic Wevers
      Oct 16 '15 at 10:12











    • I didn't expect that ArcGis can be faster that gdal. @Generic Militzer

      – Iris
      Oct 16 '15 at 10:26











    • Ah wait, this will create polygons but I need polylines.

      – Generic Wevers
      Oct 16 '15 at 10:28











    • If you put your data into a File Geodatabase it is pretty fast. But still not fast enough. That's why I'm searching for alternatives.

      – Generic Wevers
      Oct 16 '15 at 10:30






    • 2





      It's not necessarily a problem that you get polygons, you can always convert them to polylines (although, with that many it could of course take a while as well).

      – Martin
      Oct 16 '15 at 11:40














    4












    4








    4







    I'm working with R and used rasterToPolygons from the raster package in the past, but now I prefer gdal_polygonizeR by John Baumgartner. It bases on gdal_polygonize.py and is much faster.
    John Baumgartner published the code and gave an example for usage in his blog.



    If you are familiar with python you could use gdal_polygonize.py directly of course.






    share|improve this answer













    I'm working with R and used rasterToPolygons from the raster package in the past, but now I prefer gdal_polygonizeR by John Baumgartner. It bases on gdal_polygonize.py and is much faster.
    John Baumgartner published the code and gave an example for usage in his blog.



    If you are familiar with python you could use gdal_polygonize.py directly of course.







    share|improve this answer












    share|improve this answer



    share|improve this answer










    answered Oct 16 '15 at 9:46









    IrisIris

    724613




    724613








    • 1





      I'll give it I try. Last time I've used gdal_polygonize.py ArcGIS was still faster.

      – Generic Wevers
      Oct 16 '15 at 10:12











    • I didn't expect that ArcGis can be faster that gdal. @Generic Militzer

      – Iris
      Oct 16 '15 at 10:26











    • Ah wait, this will create polygons but I need polylines.

      – Generic Wevers
      Oct 16 '15 at 10:28











    • If you put your data into a File Geodatabase it is pretty fast. But still not fast enough. That's why I'm searching for alternatives.

      – Generic Wevers
      Oct 16 '15 at 10:30






    • 2





      It's not necessarily a problem that you get polygons, you can always convert them to polylines (although, with that many it could of course take a while as well).

      – Martin
      Oct 16 '15 at 11:40














    • 1





      I'll give it I try. Last time I've used gdal_polygonize.py ArcGIS was still faster.

      – Generic Wevers
      Oct 16 '15 at 10:12











    • I didn't expect that ArcGis can be faster that gdal. @Generic Militzer

      – Iris
      Oct 16 '15 at 10:26











    • Ah wait, this will create polygons but I need polylines.

      – Generic Wevers
      Oct 16 '15 at 10:28











    • If you put your data into a File Geodatabase it is pretty fast. But still not fast enough. That's why I'm searching for alternatives.

      – Generic Wevers
      Oct 16 '15 at 10:30






    • 2





      It's not necessarily a problem that you get polygons, you can always convert them to polylines (although, with that many it could of course take a while as well).

      – Martin
      Oct 16 '15 at 11:40








    1




    1





    I'll give it I try. Last time I've used gdal_polygonize.py ArcGIS was still faster.

    – Generic Wevers
    Oct 16 '15 at 10:12





    I'll give it I try. Last time I've used gdal_polygonize.py ArcGIS was still faster.

    – Generic Wevers
    Oct 16 '15 at 10:12













    I didn't expect that ArcGis can be faster that gdal. @Generic Militzer

    – Iris
    Oct 16 '15 at 10:26





    I didn't expect that ArcGis can be faster that gdal. @Generic Militzer

    – Iris
    Oct 16 '15 at 10:26













    Ah wait, this will create polygons but I need polylines.

    – Generic Wevers
    Oct 16 '15 at 10:28





    Ah wait, this will create polygons but I need polylines.

    – Generic Wevers
    Oct 16 '15 at 10:28













    If you put your data into a File Geodatabase it is pretty fast. But still not fast enough. That's why I'm searching for alternatives.

    – Generic Wevers
    Oct 16 '15 at 10:30





    If you put your data into a File Geodatabase it is pretty fast. But still not fast enough. That's why I'm searching for alternatives.

    – Generic Wevers
    Oct 16 '15 at 10:30




    2




    2





    It's not necessarily a problem that you get polygons, you can always convert them to polylines (although, with that many it could of course take a while as well).

    – Martin
    Oct 16 '15 at 11:40





    It's not necessarily a problem that you get polygons, you can always convert them to polylines (although, with that many it could of course take a while as well).

    – Martin
    Oct 16 '15 at 11:40













    4














    Try rasterToContour from the raster package.



    f <- system.file("external/test.grd", package="raster")
    r <- raster(f)
    r[r[] < 750] <- 0
    r[r[] >= 750] <- 1

    x <- rasterToContour(r)
    class(x)
    > [1] "SpatialLinesDataFrame"
    > attr(,"package")
    > [1] "sp"

    plot(r)
    plot(x, add=TRUE)


    enter image description here



    You may then easily write the files to a local folder, e.g. as 'ESRI Shapefile' (.shp), using the below code. Have a look at ogrDrivers from rgdal to find out which drivers your system is compatible with.



    library(rgdal)
    writeOGR(x, dsn = getwd(), layer = "coastlines", driver = "ESRI Shapefile")





    share|improve this answer


























    • I'll try and keep fingers crossed it will not kill my RAM. Even though I have 16GB, which hopefully is enough, R sometimes is not so efficient with big raster files. But let's see.

      – Generic Wevers
      Oct 16 '15 at 9:06











    • Conversion worked somehow, but I wasn't able to check in detail. As I am usually more into raster data processing, can you tell me how I can transfer the SpatialLineDataFrame into a shapefile or something comparable. I've googled and still struggling, as I don't know the layer name (OGRwrite).

      – Generic Wevers
      Oct 16 '15 at 10:10











    • Haha, I definitely see your point. See above update.

      – fdetsch
      Oct 16 '15 at 10:25








    • 2





      Another hint: try to set 'maxpixels' in rasterToContour to some higher value, e.g. 1e+9. You'll end up with more details then. The default setting creates quite generalized contour lines.

      – fdetsch
      Oct 16 '15 at 10:46








    • 1





      If you're not willing to resample your data to a coarser spatial resolution, the only solution I can imagine then would be to split your data into multiple tiles (e.g. 16 sub-rasters), then perform rasterToContour on each tile separately in an iterative manner and, finally, merge the resulting shapefiles into one huge shapefile. In case you are interested, our working group's package Rsenal offers a function called splitRaster to create multiple sub-rasters from one huge raster.

      – fdetsch
      Oct 16 '15 at 11:42


















    4














    Try rasterToContour from the raster package.



    f <- system.file("external/test.grd", package="raster")
    r <- raster(f)
    r[r[] < 750] <- 0
    r[r[] >= 750] <- 1

    x <- rasterToContour(r)
    class(x)
    > [1] "SpatialLinesDataFrame"
    > attr(,"package")
    > [1] "sp"

    plot(r)
    plot(x, add=TRUE)


    enter image description here



    You may then easily write the files to a local folder, e.g. as 'ESRI Shapefile' (.shp), using the below code. Have a look at ogrDrivers from rgdal to find out which drivers your system is compatible with.



    library(rgdal)
    writeOGR(x, dsn = getwd(), layer = "coastlines", driver = "ESRI Shapefile")





    share|improve this answer


























    • I'll try and keep fingers crossed it will not kill my RAM. Even though I have 16GB, which hopefully is enough, R sometimes is not so efficient with big raster files. But let's see.

      – Generic Wevers
      Oct 16 '15 at 9:06











    • Conversion worked somehow, but I wasn't able to check in detail. As I am usually more into raster data processing, can you tell me how I can transfer the SpatialLineDataFrame into a shapefile or something comparable. I've googled and still struggling, as I don't know the layer name (OGRwrite).

      – Generic Wevers
      Oct 16 '15 at 10:10











    • Haha, I definitely see your point. See above update.

      – fdetsch
      Oct 16 '15 at 10:25








    • 2





      Another hint: try to set 'maxpixels' in rasterToContour to some higher value, e.g. 1e+9. You'll end up with more details then. The default setting creates quite generalized contour lines.

      – fdetsch
      Oct 16 '15 at 10:46








    • 1





      If you're not willing to resample your data to a coarser spatial resolution, the only solution I can imagine then would be to split your data into multiple tiles (e.g. 16 sub-rasters), then perform rasterToContour on each tile separately in an iterative manner and, finally, merge the resulting shapefiles into one huge shapefile. In case you are interested, our working group's package Rsenal offers a function called splitRaster to create multiple sub-rasters from one huge raster.

      – fdetsch
      Oct 16 '15 at 11:42
















    4












    4








    4







    Try rasterToContour from the raster package.



    f <- system.file("external/test.grd", package="raster")
    r <- raster(f)
    r[r[] < 750] <- 0
    r[r[] >= 750] <- 1

    x <- rasterToContour(r)
    class(x)
    > [1] "SpatialLinesDataFrame"
    > attr(,"package")
    > [1] "sp"

    plot(r)
    plot(x, add=TRUE)


    enter image description here



    You may then easily write the files to a local folder, e.g. as 'ESRI Shapefile' (.shp), using the below code. Have a look at ogrDrivers from rgdal to find out which drivers your system is compatible with.



    library(rgdal)
    writeOGR(x, dsn = getwd(), layer = "coastlines", driver = "ESRI Shapefile")





    share|improve this answer















    Try rasterToContour from the raster package.



    f <- system.file("external/test.grd", package="raster")
    r <- raster(f)
    r[r[] < 750] <- 0
    r[r[] >= 750] <- 1

    x <- rasterToContour(r)
    class(x)
    > [1] "SpatialLinesDataFrame"
    > attr(,"package")
    > [1] "sp"

    plot(r)
    plot(x, add=TRUE)


    enter image description here



    You may then easily write the files to a local folder, e.g. as 'ESRI Shapefile' (.shp), using the below code. Have a look at ogrDrivers from rgdal to find out which drivers your system is compatible with.



    library(rgdal)
    writeOGR(x, dsn = getwd(), layer = "coastlines", driver = "ESRI Shapefile")






    share|improve this answer














    share|improve this answer



    share|improve this answer








    edited Oct 16 '15 at 10:24

























    answered Oct 16 '15 at 9:04









    fdetschfdetsch

    4,05521940




    4,05521940













    • I'll try and keep fingers crossed it will not kill my RAM. Even though I have 16GB, which hopefully is enough, R sometimes is not so efficient with big raster files. But let's see.

      – Generic Wevers
      Oct 16 '15 at 9:06











    • Conversion worked somehow, but I wasn't able to check in detail. As I am usually more into raster data processing, can you tell me how I can transfer the SpatialLineDataFrame into a shapefile or something comparable. I've googled and still struggling, as I don't know the layer name (OGRwrite).

      – Generic Wevers
      Oct 16 '15 at 10:10











    • Haha, I definitely see your point. See above update.

      – fdetsch
      Oct 16 '15 at 10:25








    • 2





      Another hint: try to set 'maxpixels' in rasterToContour to some higher value, e.g. 1e+9. You'll end up with more details then. The default setting creates quite generalized contour lines.

      – fdetsch
      Oct 16 '15 at 10:46








    • 1





      If you're not willing to resample your data to a coarser spatial resolution, the only solution I can imagine then would be to split your data into multiple tiles (e.g. 16 sub-rasters), then perform rasterToContour on each tile separately in an iterative manner and, finally, merge the resulting shapefiles into one huge shapefile. In case you are interested, our working group's package Rsenal offers a function called splitRaster to create multiple sub-rasters from one huge raster.

      – fdetsch
      Oct 16 '15 at 11:42





















    • I'll try and keep fingers crossed it will not kill my RAM. Even though I have 16GB, which hopefully is enough, R sometimes is not so efficient with big raster files. But let's see.

      – Generic Wevers
      Oct 16 '15 at 9:06











    • Conversion worked somehow, but I wasn't able to check in detail. As I am usually more into raster data processing, can you tell me how I can transfer the SpatialLineDataFrame into a shapefile or something comparable. I've googled and still struggling, as I don't know the layer name (OGRwrite).

      – Generic Wevers
      Oct 16 '15 at 10:10











    • Haha, I definitely see your point. See above update.

      – fdetsch
      Oct 16 '15 at 10:25








    • 2





      Another hint: try to set 'maxpixels' in rasterToContour to some higher value, e.g. 1e+9. You'll end up with more details then. The default setting creates quite generalized contour lines.

      – fdetsch
      Oct 16 '15 at 10:46








    • 1





      If you're not willing to resample your data to a coarser spatial resolution, the only solution I can imagine then would be to split your data into multiple tiles (e.g. 16 sub-rasters), then perform rasterToContour on each tile separately in an iterative manner and, finally, merge the resulting shapefiles into one huge shapefile. In case you are interested, our working group's package Rsenal offers a function called splitRaster to create multiple sub-rasters from one huge raster.

      – fdetsch
      Oct 16 '15 at 11:42



















    I'll try and keep fingers crossed it will not kill my RAM. Even though I have 16GB, which hopefully is enough, R sometimes is not so efficient with big raster files. But let's see.

    – Generic Wevers
    Oct 16 '15 at 9:06





    I'll try and keep fingers crossed it will not kill my RAM. Even though I have 16GB, which hopefully is enough, R sometimes is not so efficient with big raster files. But let's see.

    – Generic Wevers
    Oct 16 '15 at 9:06













    Conversion worked somehow, but I wasn't able to check in detail. As I am usually more into raster data processing, can you tell me how I can transfer the SpatialLineDataFrame into a shapefile or something comparable. I've googled and still struggling, as I don't know the layer name (OGRwrite).

    – Generic Wevers
    Oct 16 '15 at 10:10





    Conversion worked somehow, but I wasn't able to check in detail. As I am usually more into raster data processing, can you tell me how I can transfer the SpatialLineDataFrame into a shapefile or something comparable. I've googled and still struggling, as I don't know the layer name (OGRwrite).

    – Generic Wevers
    Oct 16 '15 at 10:10













    Haha, I definitely see your point. See above update.

    – fdetsch
    Oct 16 '15 at 10:25







    Haha, I definitely see your point. See above update.

    – fdetsch
    Oct 16 '15 at 10:25






    2




    2





    Another hint: try to set 'maxpixels' in rasterToContour to some higher value, e.g. 1e+9. You'll end up with more details then. The default setting creates quite generalized contour lines.

    – fdetsch
    Oct 16 '15 at 10:46







    Another hint: try to set 'maxpixels' in rasterToContour to some higher value, e.g. 1e+9. You'll end up with more details then. The default setting creates quite generalized contour lines.

    – fdetsch
    Oct 16 '15 at 10:46






    1




    1





    If you're not willing to resample your data to a coarser spatial resolution, the only solution I can imagine then would be to split your data into multiple tiles (e.g. 16 sub-rasters), then perform rasterToContour on each tile separately in an iterative manner and, finally, merge the resulting shapefiles into one huge shapefile. In case you are interested, our working group's package Rsenal offers a function called splitRaster to create multiple sub-rasters from one huge raster.

    – fdetsch
    Oct 16 '15 at 11:42







    If you're not willing to resample your data to a coarser spatial resolution, the only solution I can imagine then would be to split your data into multiple tiles (e.g. 16 sub-rasters), then perform rasterToContour on each tile separately in an iterative manner and, finally, merge the resulting shapefiles into one huge shapefile. In case you are interested, our working group's package Rsenal offers a function called splitRaster to create multiple sub-rasters from one huge raster.

    – fdetsch
    Oct 16 '15 at 11:42













    2














    While I'm a big fan of GDAL, the polygonize tool was way too slow for my applications as well.



    A fast alternative is gdal_trace_outline from Dans GDAL scripts which also has more options regarding tolerance, donuts, etc.



    Like gdal_polygonize this also produces polygons which you'd need to convert afterwards with ogr2ogr -nlt MULTILINESTRING.



    Downside to that is you need to compile it yourself, unless you are on a Linux or Mac OsX System.






    share|improve this answer
























    • Unfortunately it failed with the error message: "Segmentation fault (core dumped)". I am guessing my file is too big or more precise it will produce too many small polygons.

      – Generic Wevers
      Oct 19 '15 at 11:35
















    2














    While I'm a big fan of GDAL, the polygonize tool was way too slow for my applications as well.



    A fast alternative is gdal_trace_outline from Dans GDAL scripts which also has more options regarding tolerance, donuts, etc.



    Like gdal_polygonize this also produces polygons which you'd need to convert afterwards with ogr2ogr -nlt MULTILINESTRING.



    Downside to that is you need to compile it yourself, unless you are on a Linux or Mac OsX System.






    share|improve this answer
























    • Unfortunately it failed with the error message: "Segmentation fault (core dumped)". I am guessing my file is too big or more precise it will produce too many small polygons.

      – Generic Wevers
      Oct 19 '15 at 11:35














    2












    2








    2







    While I'm a big fan of GDAL, the polygonize tool was way too slow for my applications as well.



    A fast alternative is gdal_trace_outline from Dans GDAL scripts which also has more options regarding tolerance, donuts, etc.



    Like gdal_polygonize this also produces polygons which you'd need to convert afterwards with ogr2ogr -nlt MULTILINESTRING.



    Downside to that is you need to compile it yourself, unless you are on a Linux or Mac OsX System.






    share|improve this answer













    While I'm a big fan of GDAL, the polygonize tool was way too slow for my applications as well.



    A fast alternative is gdal_trace_outline from Dans GDAL scripts which also has more options regarding tolerance, donuts, etc.



    Like gdal_polygonize this also produces polygons which you'd need to convert afterwards with ogr2ogr -nlt MULTILINESTRING.



    Downside to that is you need to compile it yourself, unless you are on a Linux or Mac OsX System.







    share|improve this answer












    share|improve this answer



    share|improve this answer










    answered Oct 17 '15 at 11:34









    KerstenKersten

    7,27332446




    7,27332446













    • Unfortunately it failed with the error message: "Segmentation fault (core dumped)". I am guessing my file is too big or more precise it will produce too many small polygons.

      – Generic Wevers
      Oct 19 '15 at 11:35



















    • Unfortunately it failed with the error message: "Segmentation fault (core dumped)". I am guessing my file is too big or more precise it will produce too many small polygons.

      – Generic Wevers
      Oct 19 '15 at 11:35

















    Unfortunately it failed with the error message: "Segmentation fault (core dumped)". I am guessing my file is too big or more precise it will produce too many small polygons.

    – Generic Wevers
    Oct 19 '15 at 11:35





    Unfortunately it failed with the error message: "Segmentation fault (core dumped)". I am guessing my file is too big or more precise it will produce too many small polygons.

    – Generic Wevers
    Oct 19 '15 at 11:35











    0














    For posterity, I've been having success with the stars:: package in R for doing this type of operation quickly.



    library(raster)
    library(stars)
    library(sf)
    library(magrittr)

    f <- system.file("external/test.grd", package="raster")
    r <- raster(f)
    r[r[] < 750] <- 0
    r[r[] >= 750] <- 1

    x <- st_as_stars(r) %>%
    st_as_sf() %>% # this is the raster to polygons part
    st_cast("MULTILINESTRING") # cast the polygons to polylines

    plot(x)


    enter image description here



    plot(r)
    plot(x, add = TRUE)


    enter image description here





    share




























      0














      For posterity, I've been having success with the stars:: package in R for doing this type of operation quickly.



      library(raster)
      library(stars)
      library(sf)
      library(magrittr)

      f <- system.file("external/test.grd", package="raster")
      r <- raster(f)
      r[r[] < 750] <- 0
      r[r[] >= 750] <- 1

      x <- st_as_stars(r) %>%
      st_as_sf() %>% # this is the raster to polygons part
      st_cast("MULTILINESTRING") # cast the polygons to polylines

      plot(x)


      enter image description here



      plot(r)
      plot(x, add = TRUE)


      enter image description here





      share


























        0












        0








        0







        For posterity, I've been having success with the stars:: package in R for doing this type of operation quickly.



        library(raster)
        library(stars)
        library(sf)
        library(magrittr)

        f <- system.file("external/test.grd", package="raster")
        r <- raster(f)
        r[r[] < 750] <- 0
        r[r[] >= 750] <- 1

        x <- st_as_stars(r) %>%
        st_as_sf() %>% # this is the raster to polygons part
        st_cast("MULTILINESTRING") # cast the polygons to polylines

        plot(x)


        enter image description here



        plot(r)
        plot(x, add = TRUE)


        enter image description here





        share













        For posterity, I've been having success with the stars:: package in R for doing this type of operation quickly.



        library(raster)
        library(stars)
        library(sf)
        library(magrittr)

        f <- system.file("external/test.grd", package="raster")
        r <- raster(f)
        r[r[] < 750] <- 0
        r[r[] >= 750] <- 1

        x <- st_as_stars(r) %>%
        st_as_sf() %>% # this is the raster to polygons part
        st_cast("MULTILINESTRING") # cast the polygons to polylines

        plot(x)


        enter image description here



        plot(r)
        plot(x, add = TRUE)


        enter image description here






        share











        share


        share










        answered 6 mins ago









        mikoontzmikoontz

        1662




        1662






























            draft saved

            draft discarded




















































            Thanks for contributing an answer to Geographic Information Systems Stack Exchange!


            • Please be sure to answer the question. Provide details and share your research!

            But avoid



            • Asking for help, clarification, or responding to other answers.

            • Making statements based on opinion; back them up with references or personal experience.


            To learn more, see our tips on writing great answers.




            draft saved


            draft discarded














            StackExchange.ready(
            function () {
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fgis.stackexchange.com%2fquestions%2f166753%2ffastest-way-to-convert-big-raster-to-polyline-using-r-or-python%23new-answer', 'question_page');
            }
            );

            Post as a guest















            Required, but never shown





















































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown

































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown







            Popular posts from this blog

            Щит и меч (фильм) Содержание Названия серий | Сюжет |...

            is 'sed' thread safeWhat should someone know about using Python scripts in the shell?Nexenta bash script uses...

            Meter-Bus Содержание Параметры шины | Стандартизация |...