Aces High Bulletin Board
General Forums => Terrain Editor => Topic started by: JimmyD3 on March 18, 2021, 07:25:06 PM
-
Pulling map data from srtm3 seems to always come up with some strange elevation anomalies, but they are not present when using srtm30 or gtopo30. See attached elevation graphics.
(https://bbs.hitechcreations.com/smf/index.php?action=dlattach;topic=402636.0;attach=33784)
-
fixed!! mine works perfectly!
-
Okay Jaeger, here's the magic word "Please" tell me how to fix it. :D
-
LOL it was ugly!! we had to rename all the files in the download_source.txt file and that allowed it to work. easyscor link got you too the page, but then you had to rename ALL the files. if you would like me to email you the file just PM me your email address and ill get it to you
-
fixed!! mine works perfectly!
SEE why I tell you to check it in the Terrain Editor! I did warn you. :old:
The source site either changed the srtm's or the compression method changed, or both.
Kenai is exactly right, a corrugated terrain anomaly is baked into the terrain now.
I guess I need to allow that something might have went amiss with the makemap program, but that would be very surprising.
-
Googling "SRTMGL1 format" returns a page of links explaining some of the changes. The hgt extension stands for height. It has replaced the old digital elevation, dem extension.
-
easyscor is a meanie!! ok so i jumped the gun!
-
Googling "SRTMGL1 format" returns a page of links explaining some of the changes. The hgt extension stands for height. It has replaced the old digital elevation, dem extension.
Updated file links contain exactly the same files i got exactly the same file bit by bit as before update
-
The SRTM3 data generates corrugated maps. Not sure why, SRTM30 or gotop30 work fine.
-
i build most of my terrains with srtm30. if that still works i might be good then!
-
The SRTM3 data generates corrugated maps. Not sure why, SRTM30 or gotop30 work fine.
STRM3 is known to have inaccuracies - it is quite raw data while SRTM30 has lower resolution but filtered.
I assume it happens only in some ares? If you have troubles switch to 30. I'll see what I can do.
-
STRM3 is known to have inaccuracies - it is quite raw data while SRTM30 has lower resolution but filtered.
I assume it happens only in some ares? If you have troubles switch to 30. I'll see what I can do.
Okay, thanks for the update, so far I have been using SRTM30 and GOTOP30 they both work reliably.
That default elevation option would really open up other areas for use in maps as well. Again thanks for all your hard work, it makes a dramatic difference in map development and real life landscapes, especially for an old country boy like me. :D
-
Artik I'm seeing the same corrugations in the default 512 BoB output. It's even mirrored in the CBM that the Terrain Editor builds:
(https://bbs.hitechcreations.com/smf/index.php?action=dlattach;topic=402636.0;attach=33857)
-
(https://bbs.hitechcreations.com/smf/index.php?action=dlattach;topic=402636.0;attach=33865)
This has all the signs that the data has been deliberately obfuscated. The original CIA river data files had the same type of issue. This happens when the “stake holders” protect their, paid for by the USA taxpayer, work.
-
Clearly your data is corrupted.
My freshly downloaded BoB looks ok.
Can you please:
1. cleanup/move/backup content of data/srtm3 folder - such that it is empty of any previously downloaded files.
2. update your download sources from there with fixed urls: https://raw.githubusercontent.com/artyom-beilis/makeahmap/master/download_sources.txt
3. Rerun makeahmap on BoB area so it would download fresh srtm3 data.
Note, if same thing happens... try to repeat the steps with anti-virus disabled.
-
Okay, will do.
Probably tomorrow as it's late.
Oh, and keep in mind that it is the new esa data.
-
Oh, and keep in mind that it is the new esa data.
So try to use SRTM3 as is with new location
-
Same result
The first 5 lines of that download file read as follows:
srtm3/N00E006.hgt zip N00E006.hgt - http://srtm.motoroute.me/SRTM3/Africa/N00E006.hgt.zip
srtm3/N00E009.hgt zip N00E009.hgt - http://srtm.motoroute.me/SRTM3/Africa/N00E009.hgt.zip
srtm3/N00E010.hgt zip N00E010.hgt - http://srtm.motoroute.me/SRTM3/Africa/N00E010.hgt.zip
srtm3/N00E011.hgt zip N00E011.hgt - http://srtm.motoroute.me/SRTM3/Africa/N00E011.hgt.zip
srtm3/N00E012.hgt zip N00E012.hgt - http://srtm.motoroute.me/SRTM3/Africa/N00E012.hgt.zip
Wait, I forgot to empty the data folder
-
MUCH BETTER.
That data set works, of course. It still leaves the issue of the esa obfuscating the original srtm3 format, in at least 2 directions as far as I can tell in preliminary inspection.
-
MUCH BETTER.
That data set works, of course. It still leaves the issue of the esa obfuscating the original srtm3 format, in at least 2 directions as far as I can tell in preliminary inspection.
give me a link to the ESA data please?
-
https://step.esa.int/auxdata/dem/SRTMGL1/
Googling "SRTMGL1 format" returns a page of links explaining some of the changes. The hgt extension stands for height. It has replaced the old digital elevation, dem extension.
-
That fixed the corrugations on my DFW map, it looks awesome. The topography is beautiful. :x
-
https://step.esa.int/auxdata/dem/SRTMGL1/
Googling "SRTMGL1 format" returns a page of links explaining some of the changes. The hgt extension stands for height. It has replaced the old digital elevation, dem extension.
Actually it is not 3 arc second map but rather 1 arc-second accuracy map in same format.
the SRTM3 includes 1201x1201 16 bit integer numbers for each 1 by 1 degree time
this data includes 3601x3601 16 bit integer numbers for each 1 by 1 degree time that means it has much higher resolution but of course it isn't compatible with previous data that is why it didn't work for you.
Interesting maybe I can add it :headscratch:
-
Wow. That never occurred to me. Sure explains why that set didn't look right. :rofl
-
bad new, its broken again. website says they're doing updates, but its been down for 2 days.
this one has old data, but it seems to be up and running:
https://www.iea-software.com/ftp/Emeraldv6/Geography/Eurasia/Elevations/SRTM3/
-
bad new, its broken again. website says they're doing updates, but its been down for 2 days.
this one has old data, but it seems to be up and running:
https://www.iea-software.com/ftp/Emeraldv6/Geography/Eurasia/Elevations/SRTM3/
Ok... if it does not go back I'll update sources to the link you suggested.
@hitech
Maybe you can provide a mirror for these files such that community will not have to run after the new links each time one of the servers is down?
-
bad new, its broken again. website says they're doing updates, but its been down for 2 days.
this one has old data, but it seems to be up and running:
https://www.iea-software.com/ftp/Emeraldv6/Geography/Eurasia/Elevations/SRTM3/
Ohhhh...
it is different layout https://www.iea-software.com/ftp/Emeraldv6/Geography/Africa/Elevations/SRTM3/
For example before
http://srtm.motoroute.me/SRTM3/Eurasia/N00E072.hgt.zip
New
https://www.iea-software.com/ftp/Emeraldv6/Geography/Eurasia/Elevations/SRTM3/N00E072.hgt.zip
looks like they changed order of directories... need to rewrite it all :(
EDIT
The problem actually bigger since SRTM30 is down as well (same mirror) and new one does not have it.
So need to search a new mirror with all the data ASAP.
Currently GTOPO30 is only usable DB but it is much less accurate
-
Unfortunately this how the "stakeholders" protect their turf. Never mind that taxpayers pay for their equipment, including the satellites, and often their salaries through government grants when not paid directly by "state" University.
Hopefully they've updated to fill voids and corrected where they can shoot through the tree cover without format changes in the middle of each data set. I've seen that previously.
Here's hoping you find a smooth transition without too much in the way of obfuscation. :pray
-
Currently I can find online SRTM1: https://step.esa.int/auxdata/dem/SRTMGL1/ and GTOPO30
Problem SRTM1 is too large and not really needed GTOTO30 much less sharp than SRTM30 (needed for N>60)
So please find new mirrors ASAP. I assume hitech will be able to help us with storage so the data will not disappear every year...
-
I suspect you’ve already done considerable work looking for the files. Every NASA and USGS link I followed is now dead. What I did find may only be in ARC Grid or ARC ASCII format pushed by ARCGIS or ASTRA(sp?).
Probably the first thing I should tell you is that after my experience watching the USGS relocate their data every year or so, I used your program to download the entire global dataset from whichever working site was available at the time, version 2_2. I don’t remember if its zipped or not, .htg and 3.5 GB, but if you can find a site to store it, I’m willing to reinstall an ftp program and upload the folder.
I did find an interesting site, vterrain.org, with links to ftp download pages, which I copied as there is a difference between clicking their links and copying the links into a web browser. For what it might be worth...
http://vterrain.org/Elevation/SRTM/
ftp://ftp.glcf.umiacs.umd.edu/glcf/SRTM/
ftp://srtm.csi.cgiar.org/SRTM_v41/
I'm guessing you've found the CGIAR site. Nope, not looking very useful, but I don't currently have an ftp program installed to test the CGIAR links mentioned above from the vterrain.org site. (https://srtm.csi.cgiar.org/)
It looks like someone in government decided to offload handling of the data to a “community organization”, CGIAR / CIAT-CSI. If anyone else if hosting, I didn’t find it in my morning of searching. To me, it looks like name-in-lights for the organization and it’s head honcho if you use their service for non-commercial use. Like I said, that’s what it looks like to me.
Perhaps you can contact him and let him know what you’d like to do. Heck, he might hire you. They’ve apparently updated the data for shorelines and smoothed over voids. He calls it version 4.1.
The one interesting page I did find looks like it will be unusable as it seems to require manual downloads per tile, but they have the data and that ftp link might be key.
The README file of the srtm_31_05.zip file I downloaded from https://www.usna.edu/Users/oceano/pguth/md_help/html/srtm.htm says this:
The data distributed here are in ARC GRID, ARC ASCII and Geotiff format, in
decimal degrees and datum WGS84. They are derived from the USGS/NASA SRTM data.
DISTRIBUTION
Users are prohibited from any commercial, non-free resale, or redistribution
without explicit written permission from CIAT. Users should acknowledge CIAT as
the source used in the creation of any reports, publications, new data sets,
derived products, or services resulting from the use of this data set. CIAT also
request reprints of any publications and notification of any redistributing
efforts. For commercial access to the data, send requests to Andy Jarvis
(a.jarvis@cgiar.org). [NOTE: this is he same guy at CGIAR]
Once unzipped, I found the readme, a 5 line prj text file and the 886KB .asc ASCII file, started out this way:
ncols 6000
nrows 6000
xllcorner -30
yllcorner 35
cellsize 0.00083333333333333
NODATA_value -9999
-9999 -9999 -9999 -9999 -9999 -9999…
A UTube video made EarthExplorer look interesting but not particular useful for your program.
-
Interesting, I see that another folder on my HD has .gz files and it's the same size. This folder is also 3.73GB
-
Users are prohibited from any commercial, non-free resale, or redistribution
without explicit written permission from CIAT. Users
This one is very problematic... so HTC can't mirror them.
If we will not find SRTM3 I'll probably need to add a support of SRTM1... maybe convert it to SRTM3 format.
-
Keep in mind that my files were download directly from either NASA or USGS using your program, they predate CGIAR and are without CGIAR "upgrades."
No need I know of to reference them, especially as they are no longer available in this form.
Isn't there an organization that archives every public webpage ever published? I can't think of their name atm.
-
Keep in mind that my files were download directly from either NASA or USGS using your program, they predate CGIAR and are without CGIAR "upgrades."
The problem is I'm not sure their license allows to be mirrored/used.
Isn't there an organization that archives every public webpage ever published? I can't think of their name atm.
WebArchive - they don't collect zips :-( already checked
-
is it worth getting our hands on older files and just storing them on a server somewhere. as long as it works, it doesnt have to be the best resolution. id rather be able to build stuff that might not be perfect, but is buildable even if slightly flawed
-
I updated download sources with new SRTM3 mirror. You can download it from there:
https://raw.githubusercontent.com/artyom-beilis/makeahmap/master/download_sources.txt
There is still no solution for SRTM30, so if you need something above N60 you need to use GTOPO30.
-
yay thanks artik. your program is great, it just sucks that they keep changing the sources. im going to try to download all the sources and then atleast have them locally!
-
Is there any update on data sources? I can't even get GTOPO30 data now. :bhead
-
Are you sure?
I checked the link and gtopo30 seems fine. It seems we are missing only srtm30
BTW. @hitech what about the mirror?
It isn't something complicated to do. If the data will be gone I don't know what will happen with terrain generation?
Virtually all terrains now start from it
-
For 38.16N, 109.52W, using the download_sources from the link above, SRTM3 is not working for me. Am I missing something?
-
Show me the log. What the program prints?
Also need map size and scale in addition to coordinate
-
- Loading Profile from config.ini
map_size=256
map_name=canyon2
tile_set=europe
dem=srtm3
lat=38.16
lon=-109.52
scale=1.0
water_alt_limit=500
altitude_offset=-1500
- Checking for makeahmap updates... failed to get latest version number
- Version: 1.13
- Latitude, longitude range and scale
Lat: 36.306 40.014
Lon: -111.878 -107.162
Scale: 1.000
- Loading ground type mapping europe.csv
- Loading GlobCover Data.
- Loading Digital Elevations Model data.
- Downloading http://www.iea-software.com/ftp/Emeraldv6/Geography/North_America/Elevations/SRTM3/N40W112.hgt.zip
downloaded: 0.0B
Error:Failed to download file http://www.iea-software.com/ftp/Emeraldv6/Geography/North_America/Elevations/SRTM3/N40W112.hgt.zip to ./temp/tmp.zip
-
Kong, here is the latest download source file I have from artik. It has a different GTOPO30 source than what you are using.
-
Thanks Kenai. I get the same result with the download source file that you attached.
-
Dang, I guess we have to leave it with artik to resolve. Have you had any feedback from Hitech on the maps?
-
Haven't tried contacting HiTech in months. Guessing he knows that new maps are available.
-
Ok, I found. The problem was that in new mirror the layout had somewhat changed and USA was put into separate folder in comparison to the rest of North America.. (Isn't it typical of US)
I updated the download sources: https://raw.githubusercontent.com/artyom-beilis/makeahmap/master/download_sources.txt
Download them and update your local copy of download_sources.txt
It should solve the problem.
-
Works now. Thanks! :salute