LIDAR coverage expansion in progress

Boyd

Administrator
Staff member
Site Administrator
Jul 31, 2004
9,526
2,787
Ben's Branch, Stephen Creek
A major expansion of the mid-atlantic LIDAR map is currently underway. This is a huge project that I've been working on for months. Right now, the bottleneck is the amount of rendering time required, my computer is processing data 24/7 and it's going to take two or three more weeks before I'm done. My estimate is 18 million new tiles with a total size of about 500gb. This is actually very efficient, I had to download around 4,500 LIDAR DEM files from the USGS totalling about 2.4 terabytes. But lots of time-consuming processing is involved.

When completed, the entire map will be about 40 million tiles and 1.1 terabytes. Am planning to rename it "Northeast LIDAR" since it has expanded way beyond the bounds of the "Mid-Atlantic".

lidar_update.png


But for now, this is just a "heads up" because I'm going to start uploading bits and pieces to the server which might affect performance at times. There will be around 50 big "chunks" of data to upload and each one has to be individually de-compressed/installed. Not ready to make a real estimate, but hopefully it will be completed by the end of this month. Will post updates here from time to time.

When completed, there will be 2d and 3d grayscale versions in the same style as the existing Mid-Atlantic map. High resolution elevation readings will also be available through the whole area and you'll be able to view it in the terrain viewer as well. Will have to see how much disk space is left when I'm done. Am hoping there will be enough left to expand the 2023 USGS/NAIP aerials to this area as well (they now match the existing Mid-Atlantic coverage). They are used to create 3d aerials in the terrain viewer.

Beyond that, am considering adding another terabyte of disk space (which would increase the total to 3tb) when I renew the server lease at the end of the month. This would allow further expansion to the South and West to match the 24k Legacy Topo coverage area. Would like to expand the street and hybrid maps to a larger region as well.
 
Last edited:

Boyd

Administrator
Staff member
Site Administrator
Jul 31, 2004
9,526
2,787
Ben's Branch, Stephen Creek
Just now finished uploading the new Northeast US LIDAR map (2d version). Here's Cadillac Mountain on Mount Desert Island Maine, for example.

mdi.png


Note the name change, it's no longer called "Mid Atlantic LIDAR", the new name is "Northeast US LIDAR". But I've only changed the name on the menu, the actual filename(s) haven't changed so old links will still work.

menu.png


This is hot off the presses, haven't done any serious testing yet. You might have the usual cache issues with new site data, but let me know if you find other problems with the map itself.

It's going to take another day or more to upload all the elevation data that's needed for the 3d map, so you won't be able to view the new section in 3d yet. I'll update this thread when that's complete. There will also be problems with the elevation readout until the rest of the data is uploaded. The terrain viewer won't work properly for the new area until this is finished too.

Will continue to upload elevation data 24/7 until finished, so you should expect slow performance from the site in the meantime. I'll continue to post updates here until the upgrade is finished.

Looking ahead... the server lease is up for renewal at the end of the month and I just now negotiated a lower cost for another terabyte of storage, so boydsmaps will now have a total of three terabytes of disk space. That leaves lots of room for expansion - will be writing more about this soon!
 
Last edited:

Boyd

Administrator
Staff member
Site Administrator
Jul 31, 2004
9,526
2,787
Ben's Branch, Stephen Creek
FWIW, I'd say that high-resolution LIDAR data was only available for (perhaps) 70% of the new section of the map. So you'll find some areas that only have the low resolution (10-meter) legacy 1/3 arc-second DEM. Nothing I could do about that, unfortunately. We're kind of spoiled here in NJ, there are still plenty of other places with no LIDAR coverage.
 

Boyd

Administrator
Staff member
Site Administrator
Jul 31, 2004
9,526
2,787
Ben's Branch, Stephen Creek
Looks like I'm finishing this up a bit ahead of schedule. :dance: Here's the 3d version of Cadillac Mountain

Screen Shot 2024-03-29 at 10.10.06 AM.png


I haven't spent any time at all checking this out, will do that soon. In the meantime... be my guest and post any problems you encounter. I still have a number of things to do such as updating the terrain viewer and map sync apps, revising the map info and help files. Will be working on these on and off today, but they don't involve any large uploads. So - at least for now - I think the "heavy lifting" is done and the site should be running at full speed.

Looking ahead, on Monday we have tentatively scheduled a server disk upgrade which will increase total storage to 3 terabytes (from the current 2 tb). Wouldn't expect that to cause any access issues, but will post more about it later. When the upgrade is complete, I need to expand the coverage of the 2023 NAIP aerials to match the LIDAR map. That will allow you to view 3d aerials of New England in the terrain viewer. That project will involve some big uploads, but much smaller than the LIDAR map.

Anyway... we're back to normal for now, with no big uploads planned for awhile.
 
Top