relief shading for my OpenAndroMap???

[6745th@web.de]

Hi Maki,

this is great stuff!!

Can you give more details how you did it, I#d like to try on my own.

Why not just using MOBAC?

http://mobac.sourceforge.net/index.html">http://mobac.sourceforge.net/index.html

[orux]

Hi Panda, it's a bit too long to explain in detail.

You can get elevation data from http://www.viewfinderpanoramas.org/dem3.html">//http://www.viewfinderpanoramas.org/dem3.html and generate the hillshading with a GIS software, like http://qgis.org">//http://qgis.org.

Then you import the generated geotiff in https://www.mapbox.com/tilemill/">//https://www.mapbox.com/tilemill/ and export it in mbtiles format that is recognised by OruxMaps.



If you want the quick and dirty solution you are almost done. Just take a screenshot of the map in the place you like, switch without moving to the hillshade and take another screenshot. Move the two screenshots to a PC and overlay the two in Photoshop/Gimp/Krita/whatever. Put the hillshade on top and set blending mode to "multiply" and then play with opacity. This method works, but applies the shade to everything, something that I don't particularly like.



The other approach is to split you favourite theme in two: one with the background you want to shade (typically landuse), the other with the foreground (roads, buildings etc.). Take the usual screenshots and assemble accordingly.



If you want some ready to use material grab this (my theme split in two, and a couple of shaded tiles) https://app.box.com/s/p8ra7k4suy1jy0954kn4">//https://app.box.com/s/p8ra7k4suy1jy0954kn4

Hi,



I have been able to draw basic hill shadows:



http://postimage.org/">



And create a composite map, using the hill shadow map as background:



http://postimage.org/">http://postimage.org/">





But it needs more tests,



orux

[Maki]

Great, Orux!



Some observations. The map looks quite dull, because you put the hillshading below and made the foreground semitrasparent. If you reverse the things with a 20-30% opacity for the hillshading on top and 100% for the map below it gets better, but still a bit flat. If you then multiply the layers rather than just blending them the shadows are better defined and the contrast is retained (black remains black rather than turning grey). If Android standard functions don't help and you have to do the compositing by yourself, the Photoshop formula for multiply is (layer1*layer2)/256. If I have done the maths correctly the formula to multiply and lighten the shadows in one passage could be map*((hillshade*opacity)/255+1-opacity) where opacity is between 0 and 1.



To compare start from this (emulated) screenshot https://www.flickr.com/photos/30914757@N04/15572753175/">//https://www.flickr.com/photos/30914757@N04/15572753175/ and use the arrows to see the next two. The first is normal blending, the second is "multiply", the third the previous emulation I made.



As I said above this is just the "quick and dirty" way. The advantage is that it can be immediately applied to any map, the problem is that it changes the map colours significantly, unless you make the effect rather light. Changing colours for the background is the desired effect, but changing paths and roads in a non uniform way may pose problems since we relay (also) on colour to distinguish things. The best way to handle this (and the one I emulated in the third screenshot above) is to apply the shadowing only to the background. As you can see the shadow effect is much more evident, but still the paths and the tracks aren't obscured.



In theory that's easy, you just need a theme with a semi-transparent background. Any existing theme can be adapted very quickly. In practice there is a particular problem with OpenandroMaps, because they draw the sea below everything else. Making the map background transparent will just reveal the sea instead of the hillshading. Christian has surely a good reason to do that, but maybe he can change it. At a first look it seems that Freizeitkarte and standard Mapsforge maps don't have this problem, so if you want to do tests I can put together a transparent theme for that.

[orux]

Great, Orux!



Some observations. The map looks quite dull, because you put the hillshading below and made the foreground semitrasparent. If you reverse the things with a 20-30% opacity for the hillshading on top and 100% for the map below it gets better, but still a bit flat. If you then multiply the layers rather than just blending them the shadows are better defined and the contrast is retained (black remains black rather than turning grey). If Android standard functions don't help and you have to do the compositing by yourself, the Photoshop formula for multiply is (layer1*layer2)/256. If I have done the maths correctly the formula to multiply and lighten the shadows in one passage could be map*((hillshade*opacity)/255+1-opacity) where opacity is between 0 and 1.



To compare start from this (emulated) screenshot https://www.flickr.com/photos/30914757@N04/15572753175/">//https://www.flickr.com/photos/30914757@N04/15572753175/ and use the arrows to see the next two. The first is normal blending, the second is "multiply", the third the previous emulation I made.



As I said above this is just the "quick and dirty" way. The advantage is that it can be immediately applied to any map, the problem is that it changes the map colours significantly, unless you make the effect rather light. Changing colours for the background is the desired effect, but changing paths and roads in a non uniform way may pose problems since we relay (also) on colour to distinguish things. The best way to handle this (and the one I emulated in the third screenshot above) is to apply the shadowing only to the background. As you can see the shadow effect is much more evident, but still the paths and the tracks aren't obscured.



In theory that's easy, you just need a theme with a semi-transparent background. Any existing theme can be adapted very quickly. In practice there is a particular problem with OpenandroMaps, because they draw the sea below everything else. Making the map background transparent will just reveal the sea instead of the hillshading. Christian has surely a good reason to do that, but maybe he can change it. At a first look it seems that Freizeitkarte and standard Mapsforge maps don't have this problem, so if you want to do tests I can put together a transparent theme for that.

Hi, Maki,



Thanks for the information.



In the last beta I have added the option to apply hill shadows.



That version uses "multiply", it seems that the best effect is achieved.



I'll continue testing.



The first time a zone is rendered, it can be very slow, but the following times it will be faster, because the tiles with shadows are cached.





orux

[Maki]

Just tried it. Very, very promising...



First thoughts:

1) the light is wrong, it really should come from top left which is pretty much an industry standard. I know the sun is from south(-east) but that's not how our brain works. Lights and shadows are interpreted with reference to the situation in which we are. So, since light is usually coming from above if you light the map from below you reverse the perception of valleys and peaks. See http://blog.thematicmapping.org/2012/06/creating-hillshades-with-gdaldem.html">//http://blog.thematicmapping.org/2012/06/creating-hillshades-with-gdaldem.html for some nice examples.

2) the intensity is good for low zoom levels, say 11 or below. It's way too high at higher ZL. It also depends on the terrain nature, smooth hills require a more intense effect than steep mountains where things get black fast.

3) it is a lot pixellated, at every ZL. I would expect it at higher ZL, but not at ZL 13 and below. This, combined with rendering times at different zoom levels, makes me suspect that shadow tiles are recalculated at every zoom step. According to my tests rendering at ZL12 and scaling with filters at other ZL is very fast and gives a very smooth effect without pixellation.



I don't find the rendering especially slow, but given the above, and considering that a 1x1° tile can be saved in JPEG without noticeable quality loss at around 1MB, I'd explore Tobias idea of prerendering at ZL12 and scale that. I doubt people has hundreds of DEMs on the phone, and in any case it's 4% the size of the DEM itself, so no big deal.



I think that the user should have control on the shadow intensity, because the ideal setting will depend on both the map and the area (smooth vs. rugged). Maybe a quick setting in the tweaks menu. It would also be interesting to explore the possibility of automatically changing the intensity according to zoom level, in addition to what the user sets his preference. I'm doing this with buildings, for example, that are solid black when they appear and get progressively lighter when you zoom in. It helps things stand out when they are smaller, without becoming oppressive when they are bigger.



Thanks,

Maki.

[goosiebn]

Would be nice for this to be an option on any map including things like Open Cycle map and the new file maps. This would mean caching the shading by itself I suppose and blending always on the fly.

[goosiebn]

I haven't been able to post an image yet, but there's an error in the hill shading at the following co-ordinates:

54.4972N 1.0034W



If you can tell me the name of the DEM file that would be affected, and where to send it, I could e-mail you that file.



The error is a diagonal line bisecting a rectangle, with no shading in the upper left half and in the bottom right half every other line, has hill shading, the other lines do not.  The shading/no shading rows also extends the entire column of the map above and below this diagonal.

[Maki]

Hi goosiebin, you are simply at a tile border. An horizontal junction would be even worse...



Commonly available DEM data is far from perfect, there are artefacts in a lot of places and especially mismatches at tile borders. If you check OpenAndroMaps or OpenCycleMaps contour lines in the same situation you'll easily find mismatches there too (obviously since those are calculated from the same dataset).



The artefacts we are seeing in Oruxmaps are probably due to interpolation at higher zoom levels since they do not show at ZL13 and below. That's part of  the reasons why I say to render at ZL12 and just scale up and down. You can't expect photo-realistic results from DEM data and in any case hillshading doesn't have to be photo-realistic to begin with. It just serves the purpose of guiding our brain into interpreting the contours.



Also keep in mind that currently the shading is very intense, so any defect is immediately visible, with a lower opacity it would be a lot less objectionable.

[goosiebn]

Hi goosiebin, you are simply at a tile border. An horizontal junction would be even worse...



Commonly available DEM data is far from perfect, there are artefacts in a lot of places and especially mismatches at tile borders. If you check OpenAndroMaps or OpenCycleMaps contour lines in the same situation you'll easily find mismatches there too (obviously since those are calculated from the same dataset).



The artefacts we are seeing in Oruxmaps are probably due to interpolation at higher zoom levels since they do not show at ZL13 and below. That's part of  the reasons why I say to render at ZL12 and just scale up and down. You can't expect photo-realistic results from DEM data and in any case hillshading doesn't have to be photo-realistic to begin with. It just serves the purpose of guiding our brain into interpreting the contours.



Also keep in mind that currently the shading is very intense, so any defect is immediately visible, with a lower opacity it would be a lot less objectionable.

No, it's worse than that. When I get the screenshot uploaded. It's like an overflow in calculations that rolls over into subequent rows, diagonally. Up and down the column above the error some problem persists as every other row is rendered as flat, between fully hill shaded rows. It'll be an error in the DEM file that doesn't show in the regular colourised version, but due to the more complex calculations and near neghbouring data comparisons required for hill shading, this error is overflowing and becoming a huge abberation. I suppose yes the error could be a mismach at a file border, but the overflow could do with being looked at.

[goosiebn]

There are some other clusters of single pixel artifacts with very high cotrast. This is happening around negative values of elevation.

[orux]

Just tried it. Very, very promising...



First thoughts:

1) the light is wrong, it really should come from top left which is pretty much an industry standard. I know the sun is from south(-east) but that's not how our brain works. Lights and shadows are interpreted with reference to the situation in which we are. So, since light is usually coming from above if you light the map from below you reverse the perception of valleys and peaks. See http://blog.thematicmapping.org/2012/06/creating-hillshades-with-gdaldem.html">//http://blog.thematicmapping.org/2012/06/creating-hillshades-with-gdaldem.html for some nice examples.

2) the intensity is good for low zoom levels, say 11 or below. It's way too high at higher ZL. It also depends on the terrain nature, smooth hills require a more intense effect than steep mountains where things get black fast.

3) it is a lot pixellated, at every ZL. I would expect it at higher ZL, but not at ZL 13 and below. This, combined with rendering times at different zoom levels, makes me suspect that shadow tiles are recalculated at every zoom step. According to my tests rendering at ZL12 and scaling with filters at other ZL is very fast and gives a very smooth effect without pixellation.



I don't find the rendering especially slow, but given the above, and considering that a 1x1° tile can be saved in JPEG without noticeable quality loss at around 1MB, I'd explore Tobias idea of prerendering at ZL12 and scale that. I doubt people has hundreds of DEMs on the phone, and in any case it's 4% the size of the DEM itself, so no big deal.



I think that the user should have control on the shadow intensity, because the ideal setting will depend on both the map and the area (smooth vs. rugged). Maybe a quick setting in the tweaks menu. It would also be interesting to explore the possibility of automatically changing the intensity according to zoom level, in addition to what the user sets his preference. I'm doing this with buildings, for example, that are solid black when they appear and get progressively lighter when you zoom in. It helps things stand out when they are smaller, without becoming oppressive when they are bigger.



Thanks,

Maki.

Hi, Maki;



about ligth, you are right, changed to left-top.



about pixelation;



if you have DEM files with 30 mts. resolution, check your 'settings--maps--relief map--Relief map, resolution'. Try with 'high' or 'very high'.



Delete first the raster cache in 'settings--maps--reset raster cache'.





orux

[Maki]

It gets better, but it is very slow, way too slow and I still see the pixels. I tried the various settings and IMHO "medium-low" is detailed enough and plenty fast. It mostly needs to be blurred and especially toned down. Too much detail is actually detrimental, and high contrast makes pixelation more visible. I worry about the performance hit that could arise from blurring, though.

[Quercus.JB]

Just tried it. Very, very promising...



First thoughts:

1) the light is wrong, it really should come from top left which is pretty much an industry standard. I know the sun is from south(-east) but that's not how our brain works. Lights and shadows are interpreted with reference to the situation in which we are. So, since light is usually coming from above if you light the map from below you reverse the perception of valleys and peaks. See http://blog.thematicmapping.org/2012/06/creating-hillshades-with-gdaldem.html">//http://blog.thematicmapping.org/2012/06/creating-hillshades-with-gdaldem.html for some nice examples.

2) the intensity is good for low zoom levels, say 11 or below. It's way too high at higher ZL. It also depends on the terrain nature, smooth hills require a more intense effect than steep mountains where things get black fast.

3) it is a lot pixellated, at every ZL. I would expect it at higher ZL, but not at ZL 13 and below. This, combined with rendering times at different zoom levels, makes me suspect that shadow tiles are recalculated at every zoom step. According to my tests rendering at ZL12 and scaling with filters at other ZL is very fast and gives a very smooth effect without pixellation.



I don't find the rendering especially slow, but given the above, and considering that a 1x1° tile can be saved in JPEG without noticeable quality loss at around 1MB, I'd explore Tobias idea of prerendering at ZL12 and scale that. I doubt people has hundreds of DEMs on the phone, and in any case it's 4% the size of the DEM itself, so no big deal.



I think that the user should have control on the shadow intensity, because the ideal setting will depend on both the map and the area (smooth vs. rugged). Maybe a quick setting in the tweaks menu. It would also be interesting to explore the possibility of automatically changing the intensity according to zoom level, in addition to what the user sets his preference. I'm doing this with buildings, for example, that are solid black when they appear and get progressively lighter when you zoom in. It helps things stand out when they are smaller, without becoming oppressive when they are bigger.



Thanks,

Maki.

Hi, Maki;



about ligth, you are right, changed to left-top.



about pixelation;



if you have DEM files with 30 mts. resolution, check your 'settings--maps--relief map--Relief map, resolution'. Try with 'high' or 'very high'.



Delete first the raster cache in 'settings--maps--reset raster cache'.





orux

hola a todos, he realizado todo lo que indicas y este es el resultado:http://tapatalk.imageshack.com/v2/14/10/21/317b3fc87c083bf24a2c3edc8a863d52.jpg"> ¿Que puedo hacer? gracias



Enviado desde mi LG-E460 mediante Tapatalk

[goosiebn]

This is the overflow screenshot:

http://tapatalk.imageshack.com/v2/14/10/21/057dbb1c8577a1e7ca0249e999d5370b.jpg">



This is the negative height high contrast issue:

http://tapatalk.imageshack.com/v2/14/10/21/09a8b92547571d566a7fc750e362604b.jpg">

[Maki]

The overflow is the same I see at every tile border. It's much worse in mountain areas and at horizontal borders. The negative height high contrast is probably due to DEM errors, you can't do much about them. That would actually be one advantage of rendering offline on a PC, since this stuff can easily be fixed with an image editor when found. On the other side you'd have to manage multilayer maps.



I made another simulation, this time using Oruxmaps generated hillshading at "medium-low" setting. I inverted the image to compensate for the wrong lightning and rebalanced it since it become too dark. I then applied a 10 px radius gaussian blur and multiplied at around 40% opacity. Just enough to see the reliefs, not enough to screw up the colors; and no pixels in sight even at 1:1.

The flat one https://www.flickr.com/photos/30914757@N04/15592384171/in/photostream/">//https://www.flickr.com/photos/30914757@N04/15592384171/in/photostream/

The shadowed one https://www.flickr.com/photos/30914757@N04/14974311384/in/photostream/">//https://www.flickr.com/photos/30914757@N04/14974311384/in/photostream/