OruxMaps

Uncategorized Boards => MEJORAS/NEW FEATURES => Topic started by: Alejandro-FA on August 12, 2020, 11:16:21 PM

Title: More compatible DEM files extensions (such as ESRI ASCII grid)
Post by: Alejandro-FA on August 12, 2020, 11:16:21 PM
Nowadays, there are highly accurate DEM files available on the Internet (such as these: http://centrodedescargas.cnig.es/CentroDescargas/catalogo.do?Serie=LIDA2) that have a lot more precision than SRTM-DTED and GTOPO30/SRTM30, that are capped at 1" resolution.

I think it would be great for Oruxmaps to add the possibility to read those files, for better cumulative heights and other related features. Even though these type of files are a lot larger, today's smartphones have large storage capacities too.
Title: Re: More compatible DEM files extensions (such as ESRI ASCII grid)
Post by: LaurentG on August 13, 2020, 09:57:17 AM
Hi Alejandro,

you can download manually these files and copy them in the OruxMaps/Dem directory.
OruxMaps takes into account any .hgt file present in this directory, whatever the way they have been downloaded, thanks to the appli itself, or manually.

And FYI personnally, I'm using files from Sonny, that are currently probably the most accurate.
Here : http://data.opendataportal.at/dataset?res_format=hgt
These are elevation data used by OpenAndroMaps.
Title: Re: More compatible DEM files extensions (such as ESRI ASCII grid)
Post by: Alejandro-FA on August 13, 2020, 10:17:47 PM
Hi LaurentG, I think I did not made myself clear. I know that any .hgt file can be added to the OruxMaps/Dem directory. What I ask is the addition of new compatible formats. Currently, only .hgt and .dem + .hdr formats are compatible with the app, which could be better, because:


Thanks for replying.
Title: Re: More compatible DEM files extensions (such as ESRI ASCII grid)
Post by: LaurentG on August 14, 2020, 08:57:31 AM
You're right, I misunderstood your request  ;)

But are you sure that data more precise than1" exist ? I would be surprised of that !
And limitation to 1" of .hgt format is not an issue if actual data more precise is not available....

But maybe I'm wrong.

Have a look at https://www.openandromaps.org/en/more-maps-with-10m-contourlines and at http://viewfinderpanoramas.org/dem3.html

Title: Re: More compatible DEM files extensions (such as ESRI ASCII grid)
Post by: Alejandro-FA on August 16, 2020, 12:52:13 PM
Hi LaurentG, I followed your links and I think that you're confused with some concepts.

Sonny's Digital Terrain Models (https://data.opendataportal.at/dataset/dtm-spain) are made out of LiDAR data, which he has collected from public official sources. This data is indeed of higher resolution than 1" (which is approximately a 20x30 m region). BUT, he uses this data to create .hgt DEM files, which are limited to 1" resolution. This means that although the original data was way more precise, in order to distribute it with .hgt format the data has to be reduced to 1" resolution. Why you may ask? Probably because .hgt is a light-weight popular format for DEM files.

But this doesn't mean that higher resolution data can't be used with the app. As stated before, this data exists (for example, IGN has Digital Terrain Models of 5x5m and 2x2m resolution: http://centrodedescargas.cnig.es/CentroDescargas/index.jsp). This data is distributed with .asc format and, therefore, to be used with the app it should have .dem + .hdr format. I have recently discovered a post that describes how to perform this conversion easily: https://www.mendiak.net/viewtopic.php?t=58960. The problem of this data is that it requires A LOT of free storage (tens of GB). But this is going to be less of a problem in the following years, as new smartphones with more and more storage are coming.

After all, to perform the conversion is not that hard, but it would be easier if .asc files could be used directly with the app. At least, I think it would be interesting to add some sort of explanation on how to perform this conversion in the manual of OruxMaps. Thus, new users would not have such a difficult time.
Title: Re: More compatible DEM files extensions (such as ESRI ASCII grid)
Post by: LaurentG on August 16, 2020, 08:27:13 PM
Hi Alejandro,

thanks for explanations.
With your description, I agree it would be interesting to have OruxMaps reading .asc files, if these exist and are more detailed.

But I remain confused with your idea of conversion : Even if starting .asc file is 2x2m, result of conversion cannot be better than 1" resolution, if this is the best resolution for .hgt files ???

BTW, did you have set, in OruxMaps, the option to "Interpolate DEM altitudes" ? This option is a little bit "hidden" in menus, since it is under Sensors / GPS, but it works fine, and since I've set it, I'm quite "happy" with Sonny's .hgt files... even if they are not as detailed as you would expect.
Title: Re: More compatible DEM files extensions (such as ESRI ASCII grid)
Post by: Alejandro-FA on August 18, 2020, 09:16:08 AM
Hi LaurentG,

when I talk about "conversion" I mean the change of .asc format to .dem + .hdr. This last format is the only one that is both supported by oruxmaps and it is not resolution-limited, therefore, it is possible for oruxmaps to read 2 m resolution .dem + .hdr DEM files.

Btw, I do have set the "Interpolate DEM altitudes" option. As you suggest, Sonny's .hgt files are perfectly fine, and I am really happy with them too. I tried to use more precise DEM files out of curiosity because I think the SHADOW map looks gorgeous and I am trying to get height readings as accurate as possible, but in the end they are probably not that useful for most applications. Even so, I think that it is nice to have the option.
Title: Re: More compatible DEM files extensions (such as ESRI ASCII grid)
Post by: ken888 on August 28, 2020, 07:28:17 AM
The post below shows how to get accurate 5m or 2m (etc) contours from accurate .asc DEM files, and convert them to .kml/.kmz files. Oruxmaps can then overlay these accurate contours on other maps. I agree accurate DEM files would be useful in Oruxmaps to get accurate Elevation Gain stats for a track. I tried converting a .asc DEM file with 5m grid elev data with gdal_translate. But it failed because the the corner coords were not proper hgt lat lon boundaries, and the number of grid positions were not to hgt specification.
It uses gdal and GpsMapEdit. Maybe gdal can convert the gdal contour shape files so GpsMapEdit is not needed.
http://bushwalk.com/forum/viewtopic.php?f=21&t=30214
Title: Re: More compatible DEM files extensions (such as ESRI ASCII grid)
Post by: wyup on August 30, 2022, 10:44:16 PM
Quote from: Alejandro-FA on August 16, 2020, 12:52:13 PM
Sonny's Digital Terrain Models (https://data.opendataportal.at/dataset/dtm-spain) are made out of LiDAR data, which he has collected from public official sources.

Links are down, anybody can repost them?
Title: Re: More compatible DEM files extensions (such as ESRI ASCII grid)
Post by: LaurentG on August 31, 2022, 07:33:36 PM
Files are available here : https://drive.google.com/drive/folders/0BxphPoRgwhnoWkRoTFhMbTM3RDA?resourcekey=0-wRe5bWl96pwvQ9tAfI9cQg
Title: Re: More compatible DEM files extensions (such as ESRI ASCII grid)
Post by: fblc on November 12, 2022, 08:19:34 AM
And here :
https://sonny.4lima.de/
Title: Re: More compatible DEM files extensions (such as ESRI ASCII grid)
Post by: Haldo on November 25, 2022, 09:33:24 AM
Quote from: Alejandro-FA on August 13, 2020, 10:17:47 PM
Hi LaurentG, I think I did not made myself clear. I know that any .hgt file can be added to the OruxMaps/Dem directory. What I ask is the addition of new compatible formats. Currently, only .hgt and .dem + .hdr formats are compatible with the app, which could be better, because:


  • .hgt is limited to 1" resolution
  • .dem + .hdr is not limited in resolution, but it does not seem to be a popular format among free elevation data distributors. I've mostly seen GeoTiff and .asc formats for LiDar and other precise terrain elevation models. I know that they can be converted, but as far as I'm concerned it is a tricky process, at least for those that are new to GIS (like me). If someone knows of a detailed guide on how to perform the conversion to .dem + .hdr, I would appreciate it.

Thanks for replying.

Have You seen this?

https://oruxmaps.org/forum/index.php?topic=3612.msg8881#msg8881
Title: Re: More compatible DEM files extensions (such as ESRI ASCII grid)
Post by: Tronpo on November 28, 2022, 10:55:38 AM
Quote from: Alejandro-FA on August 16, 2020, 12:52:13 PM
Hi LaurentG, I followed your links and I think that you're confused with some concepts.

Sonny's Digital Terrain Models (https://data.opendataportal.at/dataset/dtm-spain) are made out of LiDAR data, which he has collected from public official sources. This data is indeed of higher resolution than 1" (which is approximately a 20x30 m region). BUT, he uses this data to create .hgt DEM files, which are limited to 1" resolution. This means that although the original data was way more precise, in order to distribute it with .hgt format the data has to be reduced to 1" resolution. Why you may ask? Probably because .hgt is a light-weight popular format for DEM files.

But this doesn't mean that higher resolution data can't be used with the app. As stated before, this data exists (for example, IGN has Digital Terrain Models of 5x5m and 2x2m resolution: http://centrodedescargas.cnig.es/CentroDescargas/index.jsp). This data is distributed with .asc format and, therefore, to be used with the app it should have .dem + .hdr format. I have recently discovered a post that describes how to perform this conversion easily: https://www.mendiak.net/viewtopic.php?t=58960. The problem of this data is that it requires A LOT of free storage (tens of GB). But this is going to be less of a problem in the following years, as new smartphones with more and more storage are coming.

After all, to perform the conversion is not that hard, but it would be easier if .asc files could be used directly with the app. At least, I think it would be interesting to add some sort of explanation on how to perform this conversion in the manual of OruxMaps. Thus, new users would not have such a difficult time.


Hola Alejandro, mira esto , aquí creo que tienes lo que buscas ,

https://tronpoonpo.blogspot.com/p/oruxdemdt05.html?m=1

y en Telegram un canal con los archivos

https://t.me/OruxDeMDT50

🅃🅁🄾🄽🄿🄾
Title: Re: More compatible DEM files extensions (such as ESRI ASCII grid)
Post by: Alejandro-FA on July 19, 2023, 01:48:15 PM
Quote from: Tronpo on November 28, 2022, 10:55:38 AM

Hola Alejandro, mira esto , aquí creo que tienes lo que buscas ,

https://tronpoonpo.blogspot.com/p/oruxdemdt05.html?m=1

y en Telegram un canal con los archivos

https://t.me/OruxDeMDT50

🅃🅁🄾🄽🄿🄾

Ostras, muchas gracias! Ahora le echo un vistazo a todo
Title: Re: More compatible DEM files extensions (such as ESRI ASCII grid)
Post by: Maki on July 21, 2023, 10:27:22 AM
Quote from: Alejandro-FA on August 12, 2020, 11:16:21 PMI think it would be great for Oruxmaps to add the possibility to read those files, for better cumulative heights and other related features. Even though these type of files are a lot larger, today's smartphones have large storage capacities too.

Storage is maybe not that much of an issue, but processing power probably is, keep in mind that this kind of data quadruples in size each time you double the resolution. Moving from 30m to 5m means having ~10x the data to read and process. ASCII grid is also uncompressed text which means gigantic files. GeoTIFF can be compressed efficiently and it's trivial to convert to/from that format, but depending on the parameters used for the compression it can be very hard to decompress (if it's not tiled you have to decompress the whole file to read it).
On top of that, using very high resolution DEMs is unlikely to be of real usefulness, because the GPS reading isn't precise enough to take advantage of it. You can find DTMs with 0.5m resolution, but they are totally pointless if you don't know exactly where you are, and you don't. Actually you will be just adding noise to your height calculations. Could be useful for stuff like slope calculation, that's true, but those are better done on a proper GIS application.
Also, using .hgt has the advantage that you avoid conflicts between files. Say for example that the user has overlapping files on one are, which one should the app use? The naming scheme in the hgt prevents that.
Your best bet, if for some reason you don't like Sonny's datasets, is to perform your own conversion to .hgt. But I'd be surprised if you find any difference.
Title: Re: More compatible DEM files extensions (such as ESRI ASCII grid)
Post by: Starfighter on December 05, 2023, 08:42:09 PM
I have read all the previous posts carefully and I would like to contribute my cent to the discussion.

I bring to the table two screen captures of the Cares River area from the Shadows virtual map made from DEM files; it is exactly the hillshading underlay layer alone. I hope this will help to understand what are we talking about when we refer to 5 m dem files.

One picture is from HGT files and the other is from HDR+DEM files obtained from MDT05 IGN .asc originals.

To use highly detailed DEM (hdr+dem) files should not be a problem but need to be indexed, which obviously are not.
Indexes in HGT files are included in its name, but HDR files must be read one by one to get the info and maybe store it in a database for quick access.

Zoom level of both pictures is 14

I would like the developer/s of Oruxmaps feel the need of review this part of the application because...

Oruxmaps is the only application out there that allows detailed hillshading!!!

Both pictures had to be cropped because of the ridiculous limit of 500KB per upload item in this forum. Without this limit I could post below 1 m precision dem files.