Author Topic: Automatic terrain generation from geographical data (a tool I created)  (Read 23220 times)

Offline Easyscor

  • Plutonium Member
  • *******
  • Posts: 10880
Re: Automatic terrain generation from geographical data (a tool I created)
« Reply #75 on: October 27, 2013, 10:16:13 PM »
You might choose to use the same river width system I developed. The width is in pixels.
// Width   Type   Count
//   5      0  (-55) x Unknown INVALID Type, there are no river type zero anymore.
//   5      1   2381 x Permanant Major River
//   4      2   4438 x Major River
//   3      3   7553 x Additional River
//   2      4   8985 x Minor River
//   1      5  (+55) x River Lake
//   2      6    246 x Double line river
//   4      7    877 x Intermittent River - major
//   3      8    977 x Intermittent River - additional
//   2      9      0 x Intermittent River - minor
//   2      10   107 x Major canal
//   2      11    93 x Lessor canal
//   0      12     0 x not used
//   2      13   242 x Irragation canal
//   0      14     0 x Users Local Stream
Ignore the counted segments. I don't think they're accurate since I've rebuilt the database for my personal use and if I remember correctly, I took the counts in the middle of that project.

It's the best publicly available database and gives great results but in addition to problems already mentioned, some of the rivers are missing.

Oh, and keep in mind that rivers change their course all the time, and new dams are built.

If I remember correctly, the river database was put together by the CIA in the '60 during the Cold War.
« Last Edit: October 27, 2013, 10:27:10 PM by Easyscor »
Easy in-game again.
Since Tour 19 - 2001

Offline artik

  • Silver Member
  • ****
  • Posts: 1907
      • Blog
Re: Automatic terrain generation from geographical data (a tool I created)
« Reply #76 on: October 28, 2013, 08:35:33 AM »
Quote
Also the whole river was the same width. I did expect this but would probably go in after your program creates the waterd.bmp file and manually taper the river at its header waters down to one pixel to give it a better aesthetic look.

Ok... this does not work with current data-set. Rivers are not continuous... It is a problem. they are constructed from several segments which makes
the process not really possible without complex topological sorting or something like that.

Take a look on this image http://cppcms.com/files/makeahmap/images/variable-river-width.png

I tried to add variation across the river length and got quite unexpected result.

Quote
I set it to river_width 4 which resulted in all rivers being 9 pixels wide on waterd.bmp. I was expecting it to be 4 pixels but what seems to have happened is that it took the original river (1 pixel wide) and then added 4 pixels to each side resulting in 9 pixels. Not sure if that is the proper behaviour / functionality but it wasn't what I expected.

It is correct but not obvious - it is border width so the size is 1+w*2 i.e. in case of 4 it is 1+4*2 =9 (explained in config.ini) I probably should change it to make more obvious.

Quote
You might choose to use the same river width system I developed. The width is in pixels.

Ok this seems to be good idea, I thought of it but hadn't implemented yet.
Artik, 101 "Red" Squadron, Israel

Offline Easyscor

  • Plutonium Member
  • *******
  • Posts: 10880
Re: Automatic terrain generation from geographical data (a tool I created)
« Reply #77 on: October 28, 2013, 09:26:55 AM »
Be sure to check the documentation. As I said, I rewrote my copy of the river database. IIRC Type 0 was originally for river segments like the  mouth of the Amazon where the segment is drawn the same way as a lake.
Easy in-game again.
Since Tour 19 - 2001

Offline ghostdancer

  • Aces High CM Staff
  • Platinum Member
  • ******
  • Posts: 7562
Re: Automatic terrain generation from geographical data (a tool I created)
« Reply #78 on: October 28, 2013, 10:52:49 AM »
No problem on the tapering of the river by the head waters. Didn't know how difficult that would be or not but as I said it is an aesthetic thing and easily something a designer can do themselves within the TE. Not to mention they already know they need to review rivers and lakes and such anyways.

Hmm, the variable width does do some unexpected things. Wonder how it looks on the TE in the TE where the rivers are rendered (whether it looks as abrupt or not as it does when looking directly at the waterd.bmp).

Yeah, I missed the description in the config.ini and just went down to the variable and the commented text there:

# Specify the width of the river in pixels
# ----------------------------------------
#
# river_width 1

Changing the text there to say or something like this would be nice for people like me that tend to skim documents. ;)

#Specify the width of the river in pixels. Following formula is used to create river width, 1+W*2
# -----------------------------------------------------------------------------------------
#
# river_width 1

X.O. 29th TFT, "We Move Mountains"
CM Terrain Team

Offline 715

  • Silver Member
  • ****
  • Posts: 1835
Re: Automatic terrain generation from geographical data (a tool I created)
« Reply #79 on: October 28, 2013, 03:00:32 PM »
I don't see the problem with the variable width rivers in artiks image.  Real rivers don't just always monotonically increase in width down stream, they get wider and narrower according to topography: i.e. wide and shallow or slow moving, and narrow and deep or fast moving.  You're always going to get "quantum" effects because rivers are such narrow things and AH's resolution is kind of coarse.  Personally I think a single line constant width river would be a bit weird looking, more like a canal.

Offline artik

  • Silver Member
  • ****
  • Posts: 1907
      • Blog
Re: Automatic terrain generation from geographical data (a tool I created)
« Reply #80 on: October 29, 2013, 05:14:28 PM »
Version 0.5 Released

Download from: http://cppcms.com/files/makeahmap/makeahmap_v0.5.zip

Bug fixes:

- Fixed river rendering to have correct width for rivers at NW/SW directions
- Fixed river width meaning now it really river width in pixels not 1+2*w
- Fixed support of spaces in path names

New Features:

- Added support of fully automatic data download - all files you need will be downloaded automatically when needed. You can disable this feature by setting download=no (see breaking changes below)
- Improver river removal algorithm such as a part of river is removed rather than entire river. You can return to old behavior by setting river_removal_policy=full.
- Added support of water depth variation near shores - you can specify a range on which a water becomes deep ocean.
- Added support of custom gndtype for the areas were water and land are mixed or not ocean depth water exits. This allows specification of custom tiles for various types of water boundaries.
- Added support of river width per river level, for example you can define river_width[2]=5 - which sets the river width of 2nd level river to 5 pixels(see breaking changes)
- By default river width changes with river majority - river level.
- Improved config.ini syntax and error reporting
- Improved error reporting in case of problems with file access

Breaking Changes

- Parameter river_width had changes its semantics
- Parameters river_water_color, land_water_color, sea_water_color and lake_water_color had been removed. User water_color[river], water_color[sea], water_color[land], water_color[lake] instead.
- Default name of globcover TIFF file had changed to "data\globcover.tif". So rename the file or change the option, otherwise it would be downloaded once again.

Known Limitations

1. The map that has area that includes the E180 - W180 meridian are not supported

2. The areas below latitude S60 are not supported. (Not much WW2 action thou)
   due to lack of the relevant data from GlobCover database

3. There is known problem of slight misalignment of the rivers and shores.
    You can try to use river_north_shift/river_east_shift options to
    amge global corrections and it is good idea to check the rivers
    ends before you start working on terrain objects.


Summary

There are many improvements - how makeahmap deals with water boundaries, water depths etc. Fully automatic file download was added to simplify program usage.

Enjoy... And report problems.

I'd really like to get feedback about water depth, gnd-types and river width changes.
Artik, 101 "Red" Squadron, Israel

Offline ghostdancer

  • Aces High CM Staff
  • Platinum Member
  • ******
  • Posts: 7562
Re: Automatic terrain generation from geographical data (a tool I created)
« Reply #81 on: October 29, 2013, 05:43:43 PM »
When you say fully automatic downloads does this also include the srtm3, srtm30, etc. elevation files? Or just the other files used to figure out land versus water, coastlines, rivers, etc.?

X.O. 29th TFT, "We Move Mountains"
CM Terrain Team

Offline Easyscor

  • Plutonium Member
  • *******
  • Posts: 10880
Re: Automatic terrain generation from geographical data (a tool I created)
« Reply #82 on: October 29, 2013, 06:44:22 PM »
I can tell you that about every year or two, the USGS has changed the address for downloading these data files. I strongly recommend that all the paths be placed in the config file instead of in the code so that they can be changed by the user in the future.
« Last Edit: October 29, 2013, 06:48:53 PM by Easyscor »
Easy in-game again.
Since Tour 19 - 2001

Offline artik

  • Silver Member
  • ****
  • Posts: 1907
      • Blog
Re: Automatic terrain generation from geographical data (a tool I created)
« Reply #83 on: October 30, 2013, 12:07:58 AM »
Quote
When you say fully automatic downloads does this also include the srtm3, srtm30, etc. elevation files? Or just the other files used to figure out land versus water, coastlines, rivers, etc.?

All of them are automatic: globcover, gshhs, gtopo30, srtm30, and even srtm3 which is very hard to download them manually.

Quote
I can tell you that about every year or two, the USGS has changed the address for downloading these data files. I strongly recommend that all the paths be placed in the config file instead of in the code so that they can be changed by the user in the future.

I already did. Every URL information is stored in the download_sources.txt. I also plan to put this particular file on the makeahmap web site so you will be able to update it separately.
Artik, 101 "Red" Squadron, Israel

Offline Easyscor

  • Plutonium Member
  • *******
  • Posts: 10880
Re: Automatic terrain generation from geographical data (a tool I created)
« Reply #84 on: October 30, 2013, 02:03:28 AM »
I hadn't noticed that but there is a problem with it, the download_sources.txt file.

When opened with Notepad, unlike your other txt files which open fine, the download_sources.txt file's endOfLine value doesn't seem to be quite right.

It opens correctly in many programs but not Notepad. I'll try to duplicate part of what I'm seeing by removing the linefeeds here:

## Format# code type file1 (file2|-) url### Globcover## http://due.esrin.esa.int/globcover/#globcover tiff - - http://due.esrin.esa.int/globcover/LandCover2009/GLOBCOVER_L4_200901_20....

instead of

#
# Format
# code type file1 (file2|-) url
#

#
# Globcover
#
# http://due.esrin.esa.int/globcover/
#

globcover tiff - - http://due.esrin.esa.int/globcover/LandCover2009/GLOBCOVER_L4_20


As you see in this example, the linefeeds are missing. I haven't looked at your code, so I can't point to the offending line but it should be easy to find and fix.

I hope this helps. You've done impressive work here in an amazingly short time.

 :cheers:
Easy in-game again.
Since Tour 19 - 2001

Offline artik

  • Silver Member
  • ****
  • Posts: 1907
      • Blog
Re: Automatic terrain generation from geographical data (a tool I created)
« Reply #85 on: October 30, 2013, 02:44:48 AM »
I hadn't noticed that but there is a problem with it, the download_sources.txt file.

When opened with Notepad, unlike your other txt files which open fine, the download_sources.txt file's endOfLine value doesn't seem to be quite right.
...

It is Notepad... The only text editor that does not handle Unix line endings (i.e. instead of Windows CR-LF only LF). Any other (normal) editor like Notepad++ knows to handle them well.

In any case I'll fix the line endings so notepad would know how to handle it.

Most of major development and preliminary testing I run on Linux, as it is much more developer friendly environment. Only than I build and do final testing that actually require TE and AH on Windows.
Artik, 101 "Red" Squadron, Israel

Offline artik

  • Silver Member
  • ****
  • Posts: 1907
      • Blog
Re: Automatic terrain generation from geographical data (a tool I created)
« Reply #86 on: October 30, 2013, 03:06:39 AM »
I also want to make an automatic clipboard map generation tool, such that nice looking maps would be created that would show both ground cover and altitude visualization, so you would be able distinguish between different areas. Current mapname.bmp is too simplistic and lacks grid.

However I don't really want to draw the grid lines and numbers manually, just too complicated - handle fonts, add a bunch of libraries, etc.

Can somebody please create plane grid images with sector numbers and sub-sector marks (i.e. numpad like markings) so I can merge them into automatically  generated map?

I need:

- Images of size 1024x1024 pixels that only contain grid and sector numbers, in any loseless  format like tiff, png or bmp (not jpeg).
- I'd like to have tham in B/W format, i.e two colors only
- I need 4 such images, for map sizes of 512, 256, 128, 64 miles.
- I need them in public domain (i.e. if you create them, just write a notices that these images in PD)

I assume there are enough map makers that can do such grids for me  :D
« Last Edit: October 30, 2013, 03:09:53 AM by artik »
Artik, 101 "Red" Squadron, Israel

Offline ghostdancer

  • Aces High CM Staff
  • Platinum Member
  • ******
  • Posts: 7562
Re: Automatic terrain generation from geographical data (a tool I created)
« Reply #87 on: October 30, 2013, 06:45:28 AM »
What email address you want them sent to?
X.O. 29th TFT, "We Move Mountains"
CM Terrain Team

Offline artik

  • Silver Member
  • ****
  • Posts: 1907
      • Blog
Re: Automatic terrain generation from geographical data (a tool I created)
« Reply #88 on: October 30, 2013, 07:40:24 AM »
artyomtnk@yahoo.com

Thanks!
Artik, 101 "Red" Squadron, Israel

Offline artik

  • Silver Member
  • ****
  • Posts: 1907
      • Blog
Re: Automatic terrain generation from geographical data (a tool I created)
« Reply #89 on: October 30, 2013, 11:42:04 AM »
There are some samples of automatically generated maps...

Battle of Britain: http://cppcms.com/files/makeahmap/images/bob.png
Israel: http://cppcms.com/files/makeahmap/images/israel.png
Black Sea: http://cppcms.com/files/makeahmap/images/black-sea.png

I used some simple grid I extracted from one of the textures, it lacks numbers. So once I get these grids I'll have the full support...
Artik, 101 "Red" Squadron, Israel