Makeahmap 1.3Alpha for AH3 terrain editor

[artik]

There are other ways to make lakes by simply anding a lake poly object (can be a simple water square) at the lake elevation. And then setting the ground elevation below the poly where you want the water to show.

HiTech

Can you please give me more details regarding the poly lake object and how it is defined

[715]

There are other ways to make lakes by simply anding a lake poly object (can be a simple water square) at the lake elevation. And then setting the ground elevation below the poly where you want the water to show.

HiTech

Do you mean creating a square object that contains a lake, much like the city, HQ, town, etc square objects?  If so, what parameter defines it as water?  High specularity?  Wave "Detail"? 

Or do you mean creating an object that is a flat plane with "lake" properties and placing it at the lakes altitude and then pushing the ground below it down to below 0 ft?  If so, that would mean the shoreline would only be able to be drawn at the relatively low 4K resolution of the altitude grid?

[hitech]

I think option 2. It gives the same as ocean beaches now. For fine detail you can always use option1.

[715]

Can you please give me more details regarding the poly lake object and how it is defined

My thoughts on this (which may or may not be useful until Hitech replies):

What Hitech calls Option 2 would be the easiest and require very minor changes to your program.  You would push all water, despite it's altitude, to below 0 altitude and you would not use your membrane technique to morph the rest of the height map; just leave it as is.  At higher altitudes this leads to ridiculously deep lakes.  To fix that you effectively create a pane of glass painted to look like water and place it, at the lakes altitude, covering the "hole".  You create this object in AC3D and import it into AH3; this would involve:
 
   a) converting a GSHHG lake to a somewhat larger diameter flat plane 3D object for input into AC3D.  Also need a shoreline "subcollide" object to keep vehicles from driving into the water 
   b) give that object a diffuse texture file the color of the lake you're working on (using some algorithm to make water near the shore not as dark?)
   c) using the Object Editor to import that object and convert it to an AH3 object
   d) create a bright (255) 8 bit bmp and assign it to the specular map and also create relevant normal map, clutter map, detail ("wavy" sand detail) map for the lake object
   e) placing that object into your terrain at the proper location and orientation and set the height to the altitude of the lake

For high altitude lakes this could lead to very steep banks unless you got perfect registration between the lake and the height map.

Option 1 would involve much more work, is very complicated, and has severe limitations that really make it only useful for "Hero" lakes.  It does, however, allow arbitrarily high resolution.  It involves creating a "Square Object" that contains a lake (or lakes).  Square Objects are squares that replace the height map terrain with a 3D object of whatever resolution you want.  They can be 1 mile, 2 miles, or 4 miles square and they must be placed in the terrain on a defined grid (not just anywhere).  Also, square objects cannot touch each other.  (This is because the underlying height map is morphed at the edges of square obects so the ground connects without ridiculous vertical walls, so you need an area to make this transition.  Two square objects adjacent to each other could have a silly boundary of vertical cliffs.)  Square Objects are what are used to give even higher resolution to the ground topology (like in the Tank Town object of CraterMA).  The process is very complex.  To understand I suggest you use the AH3 Object Editor to Export the City4x4m object to AC3D then inspect that object in AC3D.  (Also look at all the texture bmps in the export folder.)  In AC3D, and in the texture bmps, you will see two small ponds.  Turn on polygons in AC3D and open the Hierarchy Window to view the structure.  Click off the "eye" for the "low" object (that's the low resolution LOD object) so you're only looking at the high resolution object (note you still have to create both low and high resolution LODs for all objects).  You'll also see a "water" object and a "subcollide" object.  You will also see that all the objects (except subcollide) are broken into sub objects in complicated hierarcy that I don't understand at all. Somehow you'd have to auto generate all this from water body data.  Perhaps something like this:

   a) limiting yourself to lakes that fit inside a 4 mile square (and that is limited to the quantized levels that square objects are allowed, i.e. a lake could not straddle the boundary)
   b) importing SRTM data at it's highest resolution, maybe even the 1 arc sec files, to create a ground polygonal grid at arbitrarily high resolution in AC3D (I have no clue how to do this)
   c) creating a shoreline "subcollide" object to keep vehicles from driving into the water (perhaps using the SRTM ".NUM" file info?)
   d) painting a diffuse texture for the entire 4 mile patch, hopefully one that matches the colors of the normal tile textures, with the water areas made the color you want OR
   d2) painting an alpha (_A) texture that only covers the water area so the underlying textures etc get painted everywhere that isn't wet and the wet areas get the color in d)
   e) creating a specular map that makes the water shiny and a detail map with the wave detail (the sand dune detail?) on the water areas
   f) finishing the square object in AH3 Object Editor by adding trees, buildings, etc. (not necessary if you choose d2, which is probably the better way)

[artik]

@715, @HTC

I see your points, several things:

1. The proposed way of creating lakes I assume wouldn't create actual lakes that a PT boat or TG can use them. Some of the rivers and lakes had significant WW2 actions - like Dnieper, Volga, Ladoga - and you do want to have an option to run a Destroyer or a PT boat there.
2. I still need lakes to be connected with rivers - how they would interact - what happens if pt boat drives from river to lake and other way (lots of rivers contain "lake area")
3. Using custom object and running it all with object editor would significantly complicate the terrain generation from user perspective, not talking about code modification and generation of custom object by makeahmap, when currently all you need is just to import 3 files.
4. And last but not the least - (especially for selfish me ) is negative altitudes like Dead Sea or Sea of Galilee that are way below sea level and all the "flat terrain" looks very unnatural. Using the sea - level modifications I actually now recognize the relevant areas and see some distinct local features that were missing.

At higher altitudes this leads to ridiculously deep lakes.

No it does not. The technique I use tries to preserve local gradients so the modifications are very gradual, I checked several lakes at 1,500 feet altitude and they and the terrain around them look very natural (no deep canyons)

I suggest you to take a look on it - it is major difference from what I have done in makeahmap for AH2 that indeed created canyons sometimes. The global optimization approach for AH3 avoids this problem and solves some built-in issues like no negative altitudes.

Also it is good idea to put some limit to maximal altitude you allow the lakes to put - so if you have a small lake in a high mountain it wouldn't modify it too much.
There is a parameter: water_alt_limit that would remove all water bodies that their elevations is above that value.

The correct solution would be to allow water bodies to exist at all altitudes but I'm not sure it worth the effort - finally we play a game and what difference it makes a like at 0 or 1000 feet if it just - 20seconds for climb for most of planes. So as long as it looks like relevant terrain (from local perspective) I don't think it really matters weather the lakes are at 0 or at 1,000 feet altitude.

[715]

No it does not. The technique I use tries to preserve local gradients so the modifications are very gradual, I checked several lakes at 1,500 feet altitude and they and the terrain around them look very natural (no deep canyons)

You misunderstood me: I'm fully aware of your technique and impressed by it.  I was describing what one would do if one wanted lakes at altitude not equal to 0.  That is what the discussion was about.  I was trying to describe what I thought Hitech was saying.  I was saying that not morphing the surrounding terrain would lead to deep holes that could then be fixed by covering with a lake 3D "object".

All of your points are valid: for example I doubt PT boats would work and yes, the process is way more complex (that is part of what I was trying to get across). 

The obviously preferable solution is to hope Hitech finishes the river tracks and adds a lake feature (i.e. like a river but a polygon).

[artik]

Before I can do any further development I need several inputs:

1. Does the default program work for you or your fallback to CPU only version ?
2. How fast / efficient sea level optimisation works? I need the output logs of the program similar to ones I posted above.
3. Globcover Vs splat types - how well it configured? Give your inputs / adjustments of globcover.csv.


EDIT:
If you don't have time to do testing with TE just download latest version, and run it as is. It would download required files. Once it run first time I need you to run it two times more: makeahmap.exe and makeahmap_cpu.exe and provide the program output so I can see the relevant optimization performance information (copy the content of console and paste it there for regular and cpu version)

Thanks

[Oddball-CAF]

If you don't have time to do testing with TE just download latest version, and run it as is. It would download required files. Once it run first time I need you to run it two times more: makeahmap.exe and makeahmap_cpu.exe and provide the program output so I can see the relevant optimization performance information (copy the content of console and paste it there for regular and cpu version)

Thanks

Hi Artik,
  I've got -zero- experience with making maps, so I selected the above "option" to try and help out.
Here are my results:

using: makeahmap-1.5beta


(makeahmap.exe results)

- Latitude, longitude range and scale
    Lat:     27.542     34.958
    Lon:     29.433     38.107
  Scale:      4.000
- Loading GlobCover Data.
- Loading Digital Elevations Model data.
- Loading & processing shores data.
-- Total segments:1446655, maximal segment lenght:17.676
- Loading & processing rivers data...
--- Total river segments: 6649; max river segment length 1.985
  Complete
- Updateing altitudes...
-- Converting water bodies to elevations map
--- Iceles and Lakes
--- Rivers
-- Calculating elevation modifications to bring water to 0 feet
   Using:OpenCL solver running on GPU: GeForce GTX 970; computing units=13

         Grid|  Vertices| Time (s)|Iterations| Bandwidth GB/s
      256x256|     41447|    0.055|       222|    8.336
      512x512|    165367|    0.112|       380|   28.494
    1024x1024|    660538|    0.327|       667|   67.511

    Total optimization time 0.493 s
    Maximal correction  range max=2572.000 min=-1362.000
    Maximal land modificationt to below 0 alt 890
- Fixing ground types according to shorelines shapes... Done
- Saving ground types: splattype.bmp... Done
- Saving elevation data .raw and .bmp... Done
-- Maximal altitude (for use with TE bmp import) is 9790 feet
- Generating clibboard map... Done


Completed
Press Enter to exit...

==================================================================

(makeahmap_cpu.exe results)

- Latitude, longitude range and scale
    Lat:     27.542     34.958
    Lon:     29.433     38.107
  Scale:      4.000
- Loading GlobCover Data.
- Loading Digital Elevations Model data.
- Loading & processing shores data.
-- Total segments:1446655, maximal segment lenght:17.676
- Loading & processing rivers data...
--- Total river segments: 6649; max river segment length 1.985
  Complete
- Updateing altitudes...
-- Converting water bodies to elevations map
--- Iceles and Lakes
--- Rivers
-- Calculating elevation modifications to bring water to 0 feet
   Using:CPU Solver running on Intel(R) Core(TM) i7-6700K CPU @ 4.00GHz; 8 threads

         Grid|  Vertices| Time (s)|Iterations| Bandwidth GB/s
      256x256|     41447|    0.017|       222|   36.664
      512x512|    165367|    0.075|       380|   59.032
    1024x1024|    660538|    0.979|       667|   31.792

    Total optimization time 1.072 s
    Maximal correction  range max=2572.000 min=-1362.000
    Maximal land modificationt to below 0 alt 890
- Fixing ground types according to shorelines shapes... Done
- Saving ground types: splattype.bmp... Done
- Saving elevation data .raw and .bmp... Done
-- Maximal altitude (for use with TE bmp import) is 9790 feet
- Generating clibboard map... Done


Completed
Press Enter to exit...

[artik]

Hi Artik,
  I've got -zero- experience with making maps, so I selected the above "option" to try and help out.
Here are my results:

Thanks a lot, it is very valuable.

Can you please do just one more thing?

Edit the config.ini file (with notepad or any other textual editor) and change

map_size=128

in line 12 to

map_size=512

and

Under
# Israel Area
lat=31.25
lon=33.77
scale=4.0

in line 44 change the
scale=4.0

to

scale=1.0

And give me the programs output once again.

Apparently I forgot the default settings as for small terrain and I wanted to see how optimization works on biggest one.

Thanks a lot!

[Oddball-CAF]

Hi Artik,
  Here's my results after modifying the "config.ini" file per your request:

2ND TRY FOR ARTIK
(makeahmap.exe results)

- Latitude, longitude range and scale
    Lat:     27.542     34.958
    Lon:     29.433     38.107
  Scale:      1.000
- Loading GlobCover Data.
- Loading Digital Elevations Model data.
- Loading & processing shores data.
-- Total segments:1452546, maximal segment lenght:70.704
- Loading & processing rivers data...
--- Total river segments: 6651; max river segment length 7.940
  Complete
- Updateing altitudes...
-- Converting water bodies to elevations map
--- Iceles and Lakes
--- Rivers
-- Calculating elevation modifications to bring water to 0 feet
   Using:OpenCL solver running on GPU: GeForce GTX 970; computing units=13

         Grid|  Vertices| Time (s)|Iterations| Bandwidth GB/s
      256x256|     42157|    0.054|       226|    8.468
      512x512|    168163|    0.083|       290|   29.660
    1024x1024|    671751|    0.261|       547|   70.203
    2048x2048|   2685299|    1.010|       815|  107.780
    4096x4096|  10737158|    6.389|      1563|  129.176

    Total optimization time 7.796 s
    Maximal correction  range max=2125.000 min=-1368.000
    Maximal land modificationt to below 0 alt 474
- Fixing ground types according to shorelines shapes... Done
- Saving ground types: splattype.bmp... Done
- Saving elevation data .raw and .bmp... Done
-- Maximal altitude (for use with TE bmp import) is 10019 feet
- Generating clibboard map... Done


Completed
Press Enter to exit...

==============================================================

makeahmap_cpu.exe results

- Latitude, longitude range and scale
    Lat:     27.542     34.958
    Lon:     29.433     38.107
  Scale:      1.000
- Loading GlobCover Data.
- Loading Digital Elevations Model data.
- Loading & processing shores data.
-- Total segments:1452546, maximal segment lenght:70.704
- Loading & processing rivers data...
--- Total river segments: 6651; max river segment length 7.940
  Complete
- Updateing altitudes...
-- Converting water bodies to elevations map
--- Iceles and Lakes
--- Rivers
-- Calculating elevation modifications to bring water to 0 feet
   Using:CPU Solver running on Intel(R) Core(TM) i7-6700K CPU @ 4.00GHz; 8 threads

         Grid|  Vertices| Time (s)|Iterations| Bandwidth GB/s
      256x256|     42157|    0.018|       226|   38.692
      512x512|    168163|    0.059|       290|   59.186
    1024x1024|    671751|    0.829|       547|   31.416
    2048x2048|   2685299|    6.485|       815|   23.881
    4096x4096|  10737158|   50.741|      1563|   23.410

    Total optimization time 58.131 s
    Maximal correction  range max=2125.000 min=-1368.000
    Maximal land modificationt to below 0 alt 474
- Fixing ground types according to shorelines shapes... Done
- Saving ground types: splattype.bmp... Done
- Saving elevation data .raw and .bmp... Done
-- Maximal altitude (for use with TE bmp import) is 10019 feet
- Generating clibboard map... Done


Completed
Press Enter to exit...
=============================================================================

Regards, Odd

[artik]

@Oddball-CAF

Thanks a lot, it looks like even on high end graphics cards the performance of the optimizer is limited or very close to be limited by the GPU memory bandwidth. Also it seems that both holds for CPU and GPU.

Anybody with AMD/ATI Graphics Card?

[Bizman]

Radeon HD 6970

- Latitude, longitude range and scale
    Lat:     27.542     34.958
    Lon:     29.433     38.107
  Scale:      1.000
- Loading GlobCover Data.
- Loading Digital Elevations Model data.
- Loading & processing shores data.
-- Total segments:1452546, maximal segment lenght:70.704
- Loading & processing rivers data...
--- Total river segments: 6651; max river segment length 7.940
  Complete
- Updateing altitudes...
-- Converting water bodies to elevations map
--- Iceles and Lakes
--- Rivers
-- Calculating elevation modifications to bring water to 0 feet
   NOTE: No GPU OpenCL support avalible, falling back to CPU
   Using:CPU Solver running on Intel(R) Core(TM)2 Duo CPU     E8500  @ 3.16GHz;
2 threads

         Grid|  Vertices| Time (s)|Iterations| Bandwidth GB/s
      256x256|     42157|    0.078|       226|   10.546
      512x512|    168163|    0.421|       290|    8.095
    1024x1024|    671751|    4.758|       547|    5.451
    2048x2048|   2685299|   29.718|       815|    5.210
    4096x4096|  10737158|  231.911|      1563|    5.120

    Total optimization time 266.887 s
    Maximal correction  range max=2125.000 min=-1368.000
    Maximal land modificationt to below 0 alt 474
- Fixing ground types according to shorelines shapes... Done
- Saving ground types: splattype.bmp... Done
- Saving elevation data .raw and .bmp... Done
-- Maximal altitude (for use with TE bmp import) is 10019 feet
- Generating clibboard map... Done


Completed
Press Enter to exit...


================================================================================

- Latitude, longitude range and scale
    Lat:     27.542     34.958
    Lon:     29.433     38.107
  Scale:      1.000
- Loading GlobCover Data.
- Loading Digital Elevations Model data.
- Loading & processing shores data.
-- Total segments:1452546, maximal segment lenght:70.704
- Loading & processing rivers data...
--- Total river segments: 6651; max river segment length 7.940
  Complete
- Updateing altitudes...
-- Converting water bodies to elevations map
--- Iceles and Lakes
--- Rivers
-- Calculating elevation modifications to bring water to 0 feet
   Using:CPU Solver running on Intel(R) Core(TM)2 Duo CPU     E8500  @ 3.16GHz;
2 threads

         Grid|  Vertices| Time (s)|Iterations| Bandwidth GB/s
      256x256|     42157|    0.031|       226|   21.091
      512x512|    168163|    0.421|       290|    8.407
    1024x1024|    671751|    4.555|       547|    5.694
    2048x2048|   2685299|   28.673|       815|    5.398
    4096x4096|  10737158|  220.943|      1563|    5.374

    Total optimization time 254.624 s
    Maximal correction  range max=2125.000 min=-1368.000
    Maximal land modificationt to below 0 alt 474
- Fixing ground types according to shorelines shapes... Done
- Saving ground types: splattype.bmp... Done
- Saving elevation data .raw and .bmp... Done
-- Maximal altitude (for use with TE bmp import) is 10019 feet
- Generating clibboard map... Done


Completed
Press Enter to exit...

[artik]

Thanks it is very interesting,

According to the specs it supports OpenCL 1.1, the relevant Dll exist and CPU opencl support installed, but yet it fails to find opencl  gpu support. Strange...