Scaling the r-spatial ecosystem
…for the modern composable data pipeline
Dewey Dunnington (@paleolimbot)
Me
Places to find/ask/share
I’m now a low-level nuts-and-bolts developer a few years removed from day-to-day usage. If you have questions, feedback, or expertise to share there are some great venues!
- geocompx (on Discord and elsewhere)
- paleolimbot/2025-06-24_scaling-r-spatial
@paleolimboton Discord/BlueSky/Fosstodon/GitHub
I’ll collect comments/questions in the chat here
Concepts: Make your data smaller
Often data is distributed in bulk but only a small portion is needed for an analysis. Most readers will let you provide a bounding box or SQL query to limit the data that has to be dealt with in memory.
Concepts: Make your data smaller
Concepts: Make your data smaller
Concepts: Look for the spatial join
When your analysis must consider the interaction between every feature in one layer and every feature in another layer, your analysis can probably be phrased as a spatial join. Look for it! Joins are optimized in sf, databases, and elsewhere, and effectively used spatial indexes.
Concepts: Look for the spatial join
Concepts: Look for the spatial join
Concepts: Look for the spatial join
Some join examples:
Concepts: Parallel processing
Concepts: Parallel processing
Concepts: Parallel processing
Gridding 5 decades of Canadian ice coverage
- Data released weekly as shapefiles; however, format changed multiple times since the 70s
- (original format) -> .gpkg -> target attributes rasterized as separate .tif files -> packed into a grid in NetCDF files
- Rasterizing was done using the GDAL command line (run from R)
Concepts: Spatial is not special
For every person there is trying to optimize spatial there’s 50 people optimizing non-spatial. Using non-spatial tools/formats (e.g., Databases, Vanilla Data Frames, Arrow, and Parquet) can be much faster!
Concepts: Spatial is not special
For every person there is trying to optimize spatial there’s 50 people optimizing non-spatial. Using non-spatial tools/formats (e.g., Databases, Vanilla Data Frames, Arrow, and Parquet) can be much faster!
Concepts: Spatial is not special
For every person there is trying to optimize spatial there’s 50 people optimizing non-spatial. Using non-spatial tools/formats (e.g., Databases, Vanilla Data Frames, Arrow, and Parquet) can be much faster!
Concepts: Spatial is sort of special
…but also tools that special-case geometry and/or the tools you know (e.g., sf) can help you avoid mistakes and provide more compact syntax for common operations. Don’t optimize too early!
sf for big(ger) data
Beyond sf (terra, s2, geos, wk, and more)
Driving spatial databases from R
sf for big(ger) data: filter-on-read
Say you have a data source and an area you’re interested in:
sf for big(ger) data: filter-on-read
system.time({
# Read the file
elevation <- read_sf("data/ns-water_elevation.fgb")
# Transform the bounds to the CRS of the source
bounds_poly <- bounds_lonlat |>
st_transform(st_crs(elevation)) |>
st_as_sfc()
# Filter to area of interest
elevation <- elevation |>
filter(st_intersects(elevation, bounds_poly[[1]], sparse = FALSE))
})
#> user system elapsed
#> 67.656 5.454 74.056sf for big(ger) data: filter-on-read
system.time({
# Read the CRS of the source
elevation_crs <- st_layers("data/ns-water_elevation.fgb")$crs[[1]]
# Transform the bounds to the CRS of the source
bounds_poly <- bounds_lonlat |>
st_transform(elevation_crs) |>
st_as_sfc()
# Read just the data you need
elevation <- st_read(
"data/ns-water_elevation.fgb",
wkt_filter = st_as_text(bounds_poly)
)
})
#> user system elapsed
#> 7.146 0.478 7.641sf for big(ger) data: st_join()
Let’s look at the interaction of the elevation points with some lakes.
sf for big(ger) data: st_join()
If we wanted to find the mean elevation of each lake, we might do something like this:
sf for big(ger) data: st_join()
What we want here is a join!
Beyond sf: terra
Terra implements reads with extents:
…but not spatial joins (primarily targeted at raster). The API is also not based on the data.frame and converting from terra to a data frame can be expensive.
Beyond sf: wk
Beyond sf: wk
system.time({
index <- geos_strtree(elevation)
lakes$geometry |>
as_geos_geometry() |>
map_dbl(~{
elevation_candidates <- elevation[geos_strtree_query(index, .x)[[1]]]
elevations <- elevation_candidates[geos_prepared_intersects(elevation_candidates, .x)]
mean(geos_z(elevations))
})
})
#> user system elapsed
#> 2.521 0.021 2.542Driving spatial databases
Many databases provide spatial functionality but add some complications:
- How do you get data into the database?
- How do you get data out of the database?
- How do you phrase the operation you’re hoping for in SQL?
- Which DB framework (ADBC, DBI, dbplyr)
Driving spatial databases: DuckDB
Driving spatial databases: DuckDB
Driving spatial databases: DuckDB
dbGetQuery(con, "
SELECT objectid, AVG(ST_Z(elevation.geom)) FROM lakes
LEFT JOIN elevation ON ST_Intersects(lakes.geom, elevation.geom)
GROUP BY objectid
") OBJECTID avg(st_z(elevation.geom))
1 22630 208.36000
2 10909 197.13625
3 8803 195.68459
4 23875 187.48333
5 20268 207.87059
6 83574 181.70000
7 790 186.53571
8 18139 144.80909
9 44854 177.73333
10 13524 195.97667
11 50440 195.82500
12 57320 198.13333
13 51145 184.31429
14 12930 163.61765
15 101963 87.40000
16 48064 89.43867
17 67222 196.96667
18 34394 212.10000
19 4273 212.55000
20 58116 199.17150
21 45226 161.90000
22 38280 171.30000
23 85439 178.85050
24 940 187.10432
25 65311 189.14000
26 76431 198.80000
27 8323 205.88000
28 46429 226.43333
29 40378 210.17500
30 40884 198.70000
31 22143 217.05714
32 26837 222.76000
33 61314 187.55000
34 15872 187.72003
35 3287 186.01429
36 103457 186.70000
37 67980 191.70000
38 18787 202.01000
39 36018 232.72000
40 18050 237.32632
41 39797 208.20000
42 13774 211.05000
43 86873 211.26667
44 19347 201.74444
45 4925 202.40775
46 142040 196.30000
47 47652 NA
48 23006 NA
49 126628 NA
50 44780 NA
51 119716 NA
52 159592 NA
53 142844 NA
54 74704 NA
55 132963 NA
56 140494 NA
57 157856 NA
58 120807 NA
59 114174 NA
60 8439 NA
61 4374 NA
62 148171 NA
63 84349 NA
64 154033 NA
65 6518 NA
66 55270 NA
67 70717 NA
68 44726 174.33333
69 76216 181.93333
70 41751 240.83333
71 1500 205.40000
72 15884 183.53636
73 15135 171.87143
74 45778 170.86667
75 79137 195.75000
76 50579 196.25500
77 65797 189.00000
78 60865 181.80000
79 50457 185.77500
80 43973 185.50000
81 86703 107.26667
82 32819 154.02500
83 35866 180.68700
84 38632 190.20000
85 68675 183.70000
86 24203 189.30000
87 4525 204.44545
88 51655 201.25000
89 8705 198.80000
90 8767 196.50000
91 1256 197.98750
92 31369 184.07000
93 52306 179.40000
94 26983 216.65000
95 15903 231.10800
96 71108 220.90000
97 39668 213.65556
98 36188 202.52857
99 53832 195.97500
100 35 206.80000
101 44765 NA
102 42948 NA
103 55121 NA
104 105231 NA
105 136083 NA
106 150364 NA
107 136763 NA
108 106398 NA
109 89043 NA
110 25362 NA
111 66078 NA
112 94918 NA
113 131968 NA
114 109352 NA
115 36640 NA
116 154883 NA
117 77958 NA
118 115516 NA
119 24028 NA
120 22021 224.97500
121 34714 191.43500
122 8169 173.32222
123 3483 187.96832
124 79532 148.20000
125 32567 168.88333
126 74827 119.90000
127 110439 207.30000
128 112576 157.00000
129 60707 157.80000
130 113931 175.20000
131 2947 124.85000
132 33997 191.91818
133 31815 188.36250
134 77914 180.80000
135 138976 209.30000
136 15326 197.53846
137 39608 199.54000
138 75239 99.22000
139 75240 99.22000
140 70696 91.30000
141 117896 152.50000
142 67223 196.96667
143 4210 198.40000
144 15191 189.66250
145 3980 200.83610
146 101235 189.50000
147 41037 194.24444
148 63513 192.43333
149 40694 235.52000
150 60702 196.72500
151 17221 203.12105
152 85591 198.45000
153 26199 190.28333
154 16957 187.67429
155 50088 190.95000
156 98863 187.30000
157 2962 185.34000
158 77154 185.00000
159 104381 187.56100
160 101511 180.20000
161 14422 191.01176
162 29083 191.16000
163 79492 190.35000
164 33057 191.68333
165 1023 191.20000
166 43399 210.85000
167 24491 205.08333
168 36310 209.10000
169 95742 234.90000
170 63026 207.16667
171 23265 211.01250
172 32718 196.50000
173 108166 212.50000
174 5929 211.73396
175 91169 NA
176 128992 NA
177 83034 NA
178 57791 NA
179 110617 NA
180 66217 NA
181 82843 NA
182 128275 NA
183 150572 NA
184 129483 NA
185 152076 NA
186 95155 NA
187 75587 NA
188 134325 NA
189 59633 NA
190 4373 NA
191 49184 NA
192 131130 NA
193 163455 NA
194 109394 NA
195 68357 NA
196 9877 NA
197 37613 NA
198 147661 NA
199 74389 NA
200 163242 NA
201 42036 170.14000
202 38769 185.16480
203 52384 218.56667
204 75368 191.88200
205 50317 191.15800
206 40638 197.54000
207 71908 183.90000
208 791 186.53571
209 38668 174.03333
210 45149 196.90000
211 64086 184.43333
212 34834 177.40000
213 14749 195.97500
214 52404 209.74000
215 49329 234.82475
216 2464 189.40588
217 82741 195.65000
218 4772 199.40000
219 58061 182.63833
220 7798 186.61613
221 22877 161.90000
222 36198 107.00000
223 54079 107.60000
224 33471 179.13333
225 97282 173.90000
226 69810 190.60000
227 42088 175.50000
228 9086 211.92800
229 7034 199.92703
230 36281 209.43333
231 16779 211.42667
232 23363 200.83333
233 26727 197.32500
234 87616 198.60000
235 15876 187.17917
236 41577 191.11429
237 20798 212.31538
238 26184 220.52500
239 9296 230.34167
240 14482 204.02667
241 98699 202.35000
242 67375 203.95000
243 112325 NA
244 109599 NA
245 80906 NA
246 92230 NA
247 82844 NA
248 164609 NA
249 97384 NA
250 157855 NA
251 14924 NA
252 66077 NA
253 35360 NA
254 128148 NA
255 103804 NA
256 9876 NA
257 8564 NA
258 37612 NA
259 101861 NA
260 68129 NA
261 130457 NA
262 103964 NA
263 63028 170.22000
264 63429 231.10000
265 15071 198.02308
266 72733 193.11500
267 14940 172.02000
268 14939 172.02000
269 52017 173.38475
270 20156 146.73750
271 41205 148.08571
272 64085 184.43333
273 75957 174.72857
274 2936 124.85000
275 90031 195.10000
276 1 175.18235
277 31812 189.10769
278 31741 161.70000
279 51205 145.00000
280 8765 198.64375
281 51530 214.15000
282 106049 212.70000
283 29395 205.20000
284 27699 160.74444
285 42065 183.65000
286 90589 160.26400
287 33663 194.31667
288 46876 205.00000
289 46044 192.90000
290 32049 207.58750
291 84124 194.00000
292 54930 204.65000
293 79232 201.66300
294 73390 197.70000
295 1253 185.73333
296 59344 197.17500
297 19881 205.86667
298 1341 205.12249
299 20407 232.76400
300 61279 198.20000
301 82526 219.70000
302 74702 222.15000
303 36850 228.41667
304 5469 214.30000
305 7705 208.21786
306 18470 218.75455
307 31673 214.86250
308 19348 201.74444
309 16698 198.11579
310 12771 209.50000
311 83549 NA
312 59695 NA
313 106988 NA
314 72157 NA
315 63327 NA
316 26704 NA
317 105232 NA
318 44779 NA
319 127727 NA
320 96083 NA
321 147402 NA
322 123809 NA
323 105737 NA
324 120808 NA
325 119688 NA
326 130917 NA
327 69637 NA
328 49185 NA
329 59026 NA
330 129248 NA
331 96230 NA
332 106313 NA
333 93761 NA
334 20323 206.50000
335 2752 174.83167
336 103942 168.95000
337 28520 226.06667
338 742 231.37168
339 30433 155.68000
340 34787 155.70000
341 65501 149.80000
342 12465 170.09025
343 66185 178.80000
344 3910 208.00000
345 15137 171.87143
346 49204 148.35000
347 2948 124.85000
348 76239 194.00000
349 118398 193.00000
350 44594 193.46667
351 420 175.18235
352 45639 164.15000
353 25568 191.90000
354 31267 199.85482
355 90254 175.40000
356 46043 192.90000
357 24202 189.30000
358 14330 174.38750
359 24001 227.21800
360 25685 212.70000
361 1491 203.35720
362 6528 199.11892
363 19489 198.49231
364 130391 187.22300
365 3772 192.40000
366 77565 186.40000
367 16579 179.23846
368 82535 197.80000
369 15369 209.34667
370 77121 208.65000
371 52779 209.10000
372 28496 212.88182
373 71421 228.30000
374 33058 219.87500
375 10701 206.86471
376 6400 198.42647
377 55363 208.27500
378 94275 208.40000
379 33340 NA
380 66500 NA
381 126954 NA
382 79043 NA
383 123857 NA
384 146003 NA
385 151386 NA
386 121240 NA
387 37640 NA
388 154009 NA
389 76360 NA
390 114173 NA
391 103602 NA
392 109353 NA
393 74064 NA
394 82967 NA
395 12182 NA
396 132995 NA
397 6517 NA
398 29543 NA
399 148747 NA
400 145486 NA
401 41780 207.10000
402 31117 230.83333
403 29377 231.65000
404 44220 188.92500
405 5335 228.57500
406 2794 184.91327
407 45696 182.20000
408 25511 176.56667
409 90933 174.30000
410 49164 138.30000
411 34928 158.30000
412 8017 174.02414
413 7836 160.27368
414 61720 209.96667
415 91177 189.60000
416 61165 188.15000
417 152339 171.40000
418 78146 197.80000
419 28333 190.46250
420 106493 97.30000
421 22303 192.97500
422 87287 175.42857
423 68784 163.40000
424 48063 89.43867
425 65663 179.45000
426 16110 197.19600
427 14151 159.04387
428 16989 190.08000
429 69809 190.60000
430 65765 207.00000
431 52576 212.00450
432 51612 207.42500
433 50179 194.48000
434 75948 199.00000
435 98679 198.75000
436 86774 186.40000
437 40998 186.80000
438 1002 189.66000
439 130713 198.20000
440 74187 232.06667
441 97119 228.80000
442 21993 219.53333
443 5468 214.30000
444 32922 218.90000
445 128991 NA
446 44766 NA
447 21796 NA
448 13335 NA
449 56409 NA
450 63326 NA
451 101383 NA
452 108717 NA
453 34902 NA
454 81446 NA
455 130614 NA
456 106909 NA
457 95154 NA
458 117995 NA
459 105016 NA
460 130918 NA
461 55252 NA
462 13399 NA
463 28185 NA
464 140066 NA
465 21257 NA
466 38001 168.38333
467 31550 170.78333
468 99154 170.60000
469 22539 177.62500
470 25097 202.11667
471 6707 220.61818
472 18857 201.35625
473 31524 169.90247
474 30088 188.73333
475 29507 203.23333
476 93408 228.90000
477 86332 233.00000
478 3208 189.98000
479 22224 190.67500
480 78316 204.90000
481 74680 199.73333
482 19035 192.11935
483 51845 183.67500
484 32300 181.91429
485 51204 145.00000
486 89493 189.12500
487 50152 192.32500
488 40780 197.72857
489 13289 201.54167
490 44243 199.63333
491 15397 211.66667
492 33801 197.85000
493 43836 170.60000
494 77423 162.73250
495 44443 169.12500
496 32017 170.34100
497 56716 175.05000
498 46201 165.12900
499 101760 188.90000
500 3039 213.36701
501 50175 212.74000
502 25684 212.70000
503 37042 192.45000
504 2125 187.63333
505 113585 187.99200
506 29532 191.78571
507 21994 183.48889
508 80939 185.70000
509 132925 187.63500
510 71416 191.33333
511 21470 197.29091
512 29355 208.71429
513 87852 202.00000
514 138156 204.50000
515 96753 212.80000
516 13770 233.92000
517 61815 229.03333
518 69683 226.20000
519 36081 214.26667
520 32717 196.50000
521 82278 NA
522 32750 NA
523 142657 NA
524 142371 NA
525 164525 NA
526 109524 NA
527 137682 NA
528 146747 NA
529 154945 NA
530 153448 NA
531 150262 NA
532 133418 NA
533 14925 NA
534 8437 NA
535 106431 NA
536 13400 NA
537 113398 NA
538 149872 NA
539 108348 NA
540 29588 NA
541 58488 NA
542 130502 NA
543 20322 206.50000
544 33087 165.46667
545 37353 197.40500
546 13480 197.69562
547 34653 193.81150
548 116205 188.25000
549 6906 92.74634
550 48107 175.42500
551 5171 189.00000
552 66184 178.80000
553 26605 187.79091
554 9114 202.81250
555 35344 183.92500
556 26021 137.15714
557 39164 195.92000
558 122347 206.70000
559 87607 145.60000
560 127819 107.55000
561 77744 172.10000
562 98186 170.60000
563 18809 196.20455
564 10065 200.80013
565 16317 179.47846
566 16380 154.14013
567 49436 173.90000
568 77931 179.60000
569 12619 194.17300
570 18023 196.43968
571 63180 192.40000
572 14932 213.66364
573 41196 195.75550
574 22198 201.71840
575 24462 208.51429
576 43746 190.20000
577 95188 188.10000
578 61315 187.55000
579 18317 186.79500
580 147157 187.92900
581 59021 187.97467
582 137714 187.69000
583 97571 187.98400
584 7368 179.12258
585 14919 191.40000
586 60182 213.10000
587 11294 215.96000
588 34067 225.07778
589 33653 206.34000
590 83090 201.15000
591 76056 195.77500
592 10461 195.70000
593 45484 209.20000
594 71284 202.30000
595 88960 198.10000
596 10883 NA
597 42947 NA
598 47651 NA
599 112326 NA
600 23005 NA
601 79042 NA
602 126566 NA
603 34904 NA
604 156616 NA
605 67774 NA
606 120629 NA
607 102587 NA
608 100153 NA
609 120856 NA
610 109395 NA
611 15216 NA
612 29587 NA
613 155889 NA
614 9386 NA
615 75973 NA
616 132263 NA
617 55327 185.80000
618 9154 225.94783
619 6413 16.31124
620 6414 16.31124
621 42509 124.37500
622 741 231.37168
623 8434 237.88929
624 49330 195.43050
625 70190 215.66667
626 70374 179.90000
627 36154 178.23333
628 87379 176.30000
629 45628 151.85000
630 27141 213.46667
631 36608 189.30000
632 15136 171.87143
633 14223 142.07273
634 30921 195.90833
635 36436 186.80000
636 42194 202.68333
637 31740 161.70000
638 28584 166.08750
639 41007 181.12222
640 48201 203.25000
641 58458 198.70000
642 50927 198.20150
643 69365 198.65000
644 50473 171.30000
645 70981 207.75000
646 43213 186.36667
647 73685 186.60000
648 33231 179.70000
649 20961 179.50000
650 55069 178.40000
651 85672 177.80000
652 83409 196.85000
653 7722 230.63000
654 65870 218.60000
655 68712 217.20000
656 20724 213.18750
657 15993 201.64059
658 125506 200.80000
659 50372 203.70000
660 44296 NA
661 105639 NA
662 83548 NA
663 21797 NA
664 123286 NA
665 111698 NA
666 142466 NA
667 109525 NA
668 135337 NA
669 160068 NA
670 145368 NA
671 96689 NA
672 76361 NA
673 48033 NA
674 124918 NA
675 133539 NA
676 15215 NA
677 89351 NA
678 27709 NA
679 62417 162.10000
680 99153 170.60000
681 27003 223.88571
682 91393 222.80000
683 21073 193.15926
684 51323 185.36000
685 105807 222.10000
686 4035 124.73971
687 61666 177.00000
688 50665 155.36667
689 49507 162.86000
690 10518 214.91818
691 19254 195.86667
692 143462 194.70000
693 101712 172.60000
694 83532 192.60000
695 36084 193.72500
696 22875 161.90000
697 31739 161.70000
698 35903 163.17500
699 21692 107.00000
700 70697 91.30000
701 64741 153.60000
702 46340 200.05000
703 4211 198.40000
704 30148 189.75000
705 3070 163.01869
706 18928 166.12222
707 111352 177.50000
708 19002 209.16667
709 16055 198.70000
710 32275 190.84000
711 23873 183.30000
712 36798 177.55000
713 100968 176.20000
714 7864 232.95484
715 17067 217.64286
716 80552 219.60000
717 80926 223.80000
718 81681 221.10000
719 81220 211.10000
720 19142 218.66667
721 36882 205.46667
722 58301 221.65000
723 82275 205.60000
724 58798 213.70000
725 82277 NA
726 60034 NA
727 119870 NA
728 10881 NA
729 23004 NA
730 63884 NA
731 66501 NA
732 80905 NA
733 152448 NA
734 113526 NA
735 123285 NA
736 68001 NA
737 124967 NA
738 149112 NA
739 140798 NA
740 96082 NA
741 120080 NA
742 41820 NA
743 69638 NA
744 120857 NA
745 36641 NA
746 115306 NA
747 8569 231.71714
748 75453 168.80000
749 33193 196.34200
750 10127 184.94828
751 1501 205.40000
752 9231 218.45000
753 14197 139.57778
754 25999 152.16667
755 3909 208.00000
756 3832 193.40800
757 3894 192.66667
758 92807 184.60000
759 32032 198.03333
760 22876 161.90000
761 82402 182.10000
762 100492 153.30000
763 47839 236.70000
764 37581 204.65000
765 33394 178.20000
766 48447 180.40000
767 83499 174.70000
768 89782 175.15000
769 2356 200.34698
770 5529 213.78000
771 53338 206.60000
772 8324 205.88000
773 12145 206.99231
774 27331 190.56250
775 2124 187.63333
776 49786 188.16667
777 13724 187.06154
778 88569 187.61800
779 38957 186.79091
780 16803 192.16923
781 108260 199.00000
782 68140 199.80000
783 45153 227.12500
784 44088 223.64000
785 4192 206.80000
786 97381 NA
787 39963 NA
788 61723 NA
789 123395 NA
790 111697 NA
791 77990 NA
792 68000 NA
793 145395 NA
794 139090 NA
795 66076 NA
796 76869 NA
797 100154 NA
798 105017 NA
799 140518 NA
800 106430 NA
801 121228 NA
802 92729 NA
803 53190 NA
804 78862 NA
805 164322 NA
806 80233 168.80000
807 75454 168.80000
808 16565 188.19565
809 18825 208.56774
810 39317 226.61250
811 32273 186.25714
812 38654 184.91250
813 53661 187.60000
814 64022 146.75000
815 83770 186.50000
816 36469 199.63333
817 22043 198.57778
818 25445 195.93333
819 16858 172.86154
820 59424 193.20000
821 72540 199.30000
822 4773 199.40000
823 95313 192.60000
824 41278 184.00000
825 106492 97.30000
826 96439 162.90000
827 83233 188.82500
828 3553 201.73902
829 56394 202.80000
830 34998 191.85000
831 85939 186.15000
832 25566 187.70000
833 49372 178.42857
834 513 191.17642
835 57889 197.75000
836 20838 191.31408
837 1467 189.60000
838 41693 216.70000
839 41694 216.70000
840 82860 211.15000
841 17949 224.92083
842 64350 209.60000
843 61555 212.20000
844 12054 221.15900
845 68138 233.70000
846 31022 218.83750
847 38 206.80000
848 33339 NA
849 119871 NA
850 10882 NA
851 59696 NA
852 72156 NA
853 32749 NA
854 41893 NA
855 126955 NA
856 148966 NA
857 71826 NA
858 54953 NA
859 59632 NA
860 67986 NA
861 43356 NA
862 113399 NA
863 120528 NA
864 62416 162.10000
865 58695 162.10000
866 45146 225.53333
867 64714 203.43333
868 20060 125.92381
869 5419 194.77805
870 5334 228.57500
871 36465 195.05300
872 32954 171.14000
873 36948 176.40000
874 76053 159.90000
875 80296 196.15000
876 51446 195.96667
877 3025 192.66667
878 93914 199.35000
879 128734 197.60000
880 95216 193.00000
881 16852 188.03077
882 93248 174.30000
883 87606 145.60000
884 128339 139.70000
885 73397 198.80000
886 70598 177.23333
887 88253 166.20450
888 56782 188.90000
889 71172 179.76667
890 34183 211.20540
891 17381 218.57446
892 16643 199.55882
893 102148 199.20000
894 42021 191.00000
895 1353 187.28928
896 14612 210.10833
897 49280 205.43333
898 38452 223.72500
899 36311 209.10000
900 78160 228.00000
901 54310 226.02857
902 21864 210.39286
903 71432 190.00000
904 62105 208.50000
905 35319 206.26667
906 15476 206.46000
907 13334 NA
908 47918 NA
909 109598 NA
910 68124 NA
911 63885 NA
912 26705 NA
913 61722 NA
914 66216 NA
915 128274 NA
916 132403 NA
917 37639 NA
918 107670 NA
919 104592 NA
920 115428 NA
921 14775 NA
922 12664 NA
923 164841 NA
924 17152 176.00615
925 61051 133.06667
926 6559 226.87209
927 8746 185.85217
928 9232 218.45000
929 39459 228.44286
930 18409 197.06656
931 22365 207.43750
932 51840 170.91667
933 100402 173.80000
934 5172 189.00000
935 52237 196.17000
936 17592 196.01786
937 69087 205.37600
938 59754 196.02500
939 51150 195.90000
940 298 189.62712
941 61443 183.20000
942 21691 107.00000
943 101964 87.40000
944 128340 139.70000
945 44323 219.12500
946 4274 212.55000
947 15396 211.66667
948 25141 198.89183
949 85581 179.52450
950 1107 191.23569
951 58223 173.10000
952 941 187.10432
953 40964 208.06000
954 28204 198.35286
955 43306 202.48500
956 40106 211.72525
957 27629 210.35000
958 108895 198.90000
959 1257 197.98750
960 73415 186.40000
961 41082 228.46667
962 65409 180.80000
963 19958 204.28182
964 21515 204.30000
965 50200 217.75000
966 10462 195.70000
967 6445 206.48529
968 44297 NA
969 42949 NA
970 68125 NA
971 63886 NA
972 32751 NA
973 26703 NA
974 34903 NA
975 68002 NA
976 153040 NA
977 136897 NA
978 124275 NA
979 159763 NA
980 74065 NA
981 138806 NA
982 113788 NA
983 16647 NA
984 14774 NA
985 83564 NA
986 58489 NA
987 130501 NA
988 96278 165.50000
989 41921 183.70000
990 70771 228.66667
991 15983 192.31071
992 64051 176.10000
993 56398 111.55000
994 121082 147.90000
995 100731 147.40000
996 60819 189.40000
997 24137 139.85714
998 275 189.70882
999 62965 179.75000
1000 141153 192.90000
1001 30864 192.74000
1002 50004 207.45000
1003 27438 203.70071
1004 11115 213.47333
1005 15723 201.16923
1006 19340 205.09444
1007 34094 211.10000
1008 8768 196.50000
1009 35247 199.16000
1010 29820 188.68571
1011 23852 187.91091
1012 63438 224.30000
1013 47492 236.50000
1014 46151 237.36667
1015 43515 228.90000
1016 70303 213.55000
1017 17393 199.48235
1018 30343 205.91538
1019 16250 205.67143
1020 100909 213.60000
1021 62772 220.80000
1022 103596 NA
1023 123396 NA
1024 44969 NA
1025 139388 NA
1026 130553 NA
1027 162900 NA
1028 72452 NA
1029 25363 NA
1030 25364 NA
1031 76359 NA
1032 78694 NA
1033 8438 NA
1034 13502 NA
1035 130294 NA
1036 80264 NA
1037 106945 NA
1038 124917 NA
1039 103803 NA
1040 108210 NA
1041 8563 NA
1042 92730 NA
1043 53891 NA
1044 93762 NA
1045 133461 NADriving spatial databases: DuckDB
library(dbplyr)
library(geoarrow)
tbl(con, sql("SELECT geom FROM lakes LIMIT 5")) |>
arrow::to_arrow() |>
st_as_sf()Simple feature collection with 5 features and 0 fields
Geometry type: MULTIPOLYGON
Dimension: XYZ
Bounding box: xmin: 369038.9 ymin: 4965937 xmax: 371900.8 ymax: 4968380
z_range: zmin: 204.4 zmax: 221.4
CRS: NA
geom
1 MULTIPOLYGON Z (((370253.3 ...
2 MULTIPOLYGON Z (((370253.3 ...
3 MULTIPOLYGON Z (((371068.9 ...
4 MULTIPOLYGON Z (((371136.9 ...
5 MULTIPOLYGON Z (((371900.7 ...Driving spatial databases: PostGIS
Driving spatial databases: PostGIS
con |>
execute_adbc("CREATE TABLE lakes (objectid float8, geometry GEOMETRY)")
con |>
execute_adbc("CREATE TABLE elevation (geometry GEOMETRY)")
lakes |>
as_tibble() |>
transmute(objectid = OBJECTID, geometry = wk::as_wkb(geometry)) |>
write_adbc(con, "lakes", mode = "append")
tibble(geometry = elevation |> wk::as_wkb()) |>
write_adbc(con, "elevation", mode = "append")Driving spatial databases: PostGIS
Create the index:
Driving spatial databases: PostGIS
Run the join!
con |>
read_adbc("
SELECT objectid, AVG(ST_Z(elevation.geometry)) FROM lakes
LEFT JOIN elevation ON ST_Intersects(lakes.geometry, elevation.geometry)
GROUP BY objectid
") |>
as_tibble() |>
head(5)# A tibble: 5 × 2
objectid avg
<dbl> <dbl>
1 62417 162.
2 36311 209.
3 36436 187.
4 33339 NA
5 3980 201.Driving spatial databases: Apache Sedona
Check out the Sedona R Package documentation
Scaling r-spatial
Acknowledgements
- Edzer Pebezma (sf)
- Michael Sumner (hypertidy, vapour)
- Josiah, Terri, Jesse @ R Consortium
- Hadley Wickham
- Contributors to r-spatial old and new!