Messages

[HGT.zip]

Hi



to save up space on the SDcard, oruxmaps could support reading HGT.zip (or .gz, .Z ...) files.



It may not be relevant, but the SRTMGHGT driver from gdal supports it:

http://www.gdal.org/frmt_various.html">http://www.gdal.org/frmt_various.html



thanks again, this software is awesome.



yip

[if Oruxmaps were to map "Media Next" and "Media Previous" special android codes to next /previous map provider (off/on-line)]

Hi all



The idea here would be to map physical device keys, like vol+/vol-/camera shutter ... to predefined actions in OruxMaps.



For example, a long press on vol +/- would cycle through the available maps.

I spend a lot of time going back and forth switching maps providers, and this would much more simple.



About the technical implementation,

if Oruxmaps were to map "Media Next" and "Media Previous" special android codes to next /previous map provider (off/on-line)

that would be a total blast.



MediaPrevious and MediaNext are then easily controlled by Xposed module "Xposed additions", for example like a long press on vol+/- (or anything).



and the Camera key (for devices that support it) could be Start/Stop GPS tracking for example.



Well, the idea here is to map the different android defined keys, wether they really exist on the device or not, to Oruxmaps features.

It shouldn't affect the vast majority of users who probably don't bother, but will allow power users to control OM with ease, even in the rain with a messy touchscreen, one-handed ...



Okay, thanks for reading



yip

[the ASTER GDEM v2 were impacted by this issue.]

thanks Maki !



I've modified the script to fix this quotes issue; there was a typo there also.

There was also a problem with Geotiff sized 3601 , 7201 ... instead of 3600 ... 7200...

should be fixed now. the ASTER GDEM v2 were impacted by this issue.



and I'm glad to know it worked for custom geotiff as well !

[GDAL tools]

--> Or how to get the Europe DEM into Oruxmaps.



(Please excuse my English)

Hello

For those of us who doesn-t live in Andalucia and cannot enjoy a 5mx5m DEM we need to rely on global DEM sources.

Some years ago, I wrote a tutorial to port ASTER GDEM (30mx30m) to OruxMaps (http://oruxmaps.forosactivos.net/t2757-altitude-and-3d-with-aster-gdem-tutorial">//http://oruxmaps.forosactivos.net/t2757-altitude-and-3d-with-aster-gdem-tutorial)



Recently, Europe has released a comparable dataset, EUDEM (also based on ASTER  :lol: ):

http://www.eea.europa.eu/data-and-maps/data/eu-dem">//http://www.eea.europa.eu/data-and-maps/data/eu-dem , download the ones you want.



Here is a tool to convert these files to HGT so you can use them in Oruxmaps.

-install GDAL tools

-copy the following snippet in a new file and name it "tif2hgt.bat"

-configure line 17 of the script with your GDAL install path

-in Windows, drag&drop (multiple) tif files on the script, they will be converted to HGT.

Code Select Expand


rem
rem  TIF 2 HGT
rem
rem Purpose: convert TIF DEM into HGT DEM files (Digital Elevation Model)
rem
rem Usage: Drag&Drop multiple TIFs file on this script (but NO directory!)
rem
rem Prerequisites : GDAL utilities and a bit of configuration below
rem warning : no error checking !
rem warning : AVOID whitespaces/specialchars in filenames AND working paths
rem author: yip 15 june 2016

@echo off
setlocal enableextensions enabledelayedexpansion

rem CONFIGURATION
set gdalpath="c:Program FilesGDAL"
rem END CONFIG

rem should not be changed. HGT is almost always this size. only deal with square HGTs. I don't know about poles.
set hgtsize=3600
rem dirty hack for the FOR loops
set /A hgtone=%hgtsize%+1

rem only for displaying progress
for %%z in (%*) do set /A totalfiles+=1

rem MAIN LOOP
rem process each drag&dropped file
for %%F in (%*) do (

set /A currentfile+=1
echo ========Processing [%%~nxF]========

echo [Phase 0] Get info about the TIF with GDAL ...
rem get the size of the tif (eg. 18000x18000)
%gdalpath%gdalinfo.exe %%F | findstr /c:"Size is" > tempdem.out
set /p tifsize=<tempdem.out
for /f "tokens=3,4 delims=, " %%g in (tempdem.out) do (
set xtifsize=%%g & set /A xmax=%%g-2
set ytifsize=%%h & set /A ymax=%%h-2
)
set /A totalsize= !xtifsize! / %hgtsize% * !ytifsize! / %hgtsize%

rem loop for each row of that TIFF
for /l %%X in (0, %hgtsize% , !xmax!) do (

rem loop for each column of that TIFF
for /l %%Y in (0, %hgtsize% , !ymax!) do (

echo [Phase 1] Gdal-translate for window %%X %%Y ,ignore HGT name error ...
%gdalpath%gdal_translate.exe -strict -q -r cubic -eco -of SRTMHGT -srcwin %%X %%Y %hgtone% %hgtone% %%F N99E999.HGT 2>&1 | findstr "Expected" > tempdem.out
set /p goodhgt=<tempdem.out
set goodhgt=!goodhgt:~-12,-1!

echo [Phase 2] rename N99E999.HGT to !goodhgt! and clean the temp files ...
rename N99E999.HGT !goodhgt! && del N99E999.HGT.aux.xml tempdem.out

set /A progress+=1
echo --Done--[!goodhgt!]-[HGT:!progress!/!totalsize!]-[TIF:!currentfile!/%totalfiles%]-----
@title tif2hgt : [TIF:!currentfile!/%totalfiles%] [HGT:!progress!/!totalsize!] Processing ... [%%~nxF]
)
)
)
echo =========== END ( check your files, NO error checking ! ) ==========
@title tif2hgt : END
pause


It should work on Windows 7+. Yes it is a dirty little script. I take no responsability if something goes wrong.



About the mentionned EU-DEM dataset, each TIF transforms into 25 HGT. the whole europe 20GB tif give birth to 4000 .HGT (100GB) . each HGT is 25MB.

Be sure to have loads of free space on your device !!



(About my considerations about HGT format among the other DEMS, see: http://www.oruxmaps.com/foro/viewtopic.php?f=7&t=3611">//http://www.oruxmaps.com/foro/viewtopic.php?f=7&t=3611)



yip



edit : 15 june 2016 :

-"Programs Files" to "Program Files"

-add quotes around Program Files, thanks Maki !

- changed boundaries from TiffSize - 1 to TiffSize -2 to address an issue with ASTER GDEM. I would not be surprised if the HGT generated would be shifted right 1 pixel as a side effect now...

[orux]

Hi all,

Firstly, sorry about the language, english is not my primary language.



Some time ago, orux added Elevation data features in the app.

If not using the online option, the app needs the DEM data files to be in either of these formats:

• HGT

• DEM+HDR

(which I believe is BIL+HDR)[/list]

To better understand them,

Here I will sum up the main differences between the two (as I see them 'cause I'm no expert), and I'll add the Geotiff which is also pretty standard. I'm sure people here knows a lot better, feel free to comment.



HGT:

• -

Coverage: 1 file for each 1° x 1° tile[/list]

• -

Resolution: fixed, either 1 arc/s or 3 arc/s (30m or 90m)[/list]

• -

Vertical Precision: from -32000m to 32000m, in steps of 1m[/list]

• -

Data Size: 25MB, only uncompressed (for 1 arc/s file)[/list]

DEM+HDR:

• -

Coverage: free size file (described in the HDR file)[/list]

• -

Resolution: free I think[/list]

• -

Vertical Precision: from -32000m to 32000m, in steps of 1mm (millimeter) probably[/list]

• -

Data Size: 50MB, only uncompressed (for 1 arc/s file)[/list]

GeoTiff:

• -

Coverage: free size file (described in the file itself)[/list]

• -

Resolution: free I think[/list]

• -

Vertical Precision: from -32000m to 32000m(?), in steps of 1mm (millimeter) probably[/list]

• -

Data Size: ~5MB, compressed, 50MB uncompressed (for 1 arc/s file)[/list]

In other words, it seems that there is a lot of bloat in GeoTiff and DEM/BIL formats, just to retain this extra vertical precision that is not so reliable anyway, depending on the data source.

and that explains why a set of (compressed) HGT files is always smaller than its original GeoTiff source at the same resolution.



Based on these thoughts, for me HGT is the clear winner. it takes half the size of DEM+HDR, but carry the same precision; except for the vertical resolution but most people would'nt need a millimeter precision for their outdoor activities ...

However, It has drawbacks:

-It is not available for resolution better that 30m (1 arc/sec)

-It is not compressed (but neither do the DEM+HDR format)



Well, I'm not certain of all that, please feel free to add your own experience on the matter  :oops:



yip

orux can read mbtiles, why do you want to convert?

mbtiles format doesn't allow a custom projection

rmap format does



oruxmaps can apply a custom datum via the "maps" menu to rmaps, but not to mbtiles.

(this option could perhaps be removed for unsupported maps then ?)

[If the menu "Format Convert" -> "Online maps ..." doesn't launch anything when I click on it:]

PBS, Portable Basemap Server is a software introduced by its author 2 years ago, on the oruxmaps forums :

http://oruxmaps.forosactivos.net/t3350-a-tool-for-downloading-mbtiles-map-alternative-to-mobac">//http://oruxmaps.forosactivos.net/t3350-a-tool-for-downloading-mbtiles-map-alternative-to-mobac



Small & efficient software, that does the job nicely.

But documentation is scarce.

Here is what I've discovered that may be of interest to some of the oruxmaps users:



If the menu "Format Convert" -> "Online maps ..." doesn't launch anything when I click on it:

Looks like a bug. It may happen if "Local IP Address" (in the main PBS window) is set to an IPv6 like "::1". try to set it back to another IP.



Window "Convert Online Maps to MBtiles":



* Always set "RecreateEmptyTiles", so that the creation of the database is faster when resuming a download.

* Don't tick "Compact ?" unless necessary. in most cases, it put a heavy load on the CPU, for very small gain.



* Limit the "download extend" to the target area by using "import shapefile", instead of draw-extent-by-mouse.

In this context, a shapefile is like a polygon area.

less bandwith, less space on disk, less stress on the servers for tiles you don't care about, and faster creation time.

It's quite easy to create a shapefile when you know the trick; see this sites: http://www.mapmash.in/kmlpolyline.html">//http://www.mapmash.in/kmlpolyline.html & http://www.zonums.com/online/kml2shp.php">//http://www.zonums.com/online/kml2shp.php for creating one online, without any additionnal software.



* map creation may be slow at times, and PBS can spend a lot of time "writing to disk".

It is because for some reason,the bigger the map, the longer it takes to append new tiles to it.

So Basically, it all boiled down to the hard drive speed. a SSD would probably be a lot faster than a old 5400rpm HDD.

To speed things up, I used a ramdisk and made PBS write directly to it. it decrease the map creation time tremendously! Once finished, I just need to copy the mbtiles from the ramdisk to the harddrive so it doesn't vanish at power-off.

having 8Gb of RAM, I created a ramdisk of max 5GB, windows swap is kicking in smoothly if needed anyway so it's rather safe.

Software used for ramdisk creation]old sqlite cannot support optimizations[/b] during the creation of big maps.

These optimizations have been in the official releases of sqlite for 5 years now, but this driver pre-date that.

(It appears that MOBAC don't use them neither, while probably using a more recent SQLite framework)



It is a pity because sqlite-writing seems to be on of the main causes of trouble with both software.



If someone has enough knowledge to recompile it with up-to-date libs,

It may be interesting to try these 2 SQLite settings:

PRAGMA synchronous=NORMAL;

PRAGMA journal_mode=WAL;

yip

[So what is the exact, and minimal, structure of RMAP open format, that Oruxmaps can parse ?]

Hi



I recently tried to convert mbtiles to rmaps.

AFAIK , RMAP file format covers numerous formats, some open, some proprietary & closed specs ...



I would like to talk here about the open, sqlite, simple RMAP format. not the twonav/swiss/compegps/(...) ones.

How is it structured ? information is very scarce on the net. even on the rmaps dev site (if related anyway).

So far , I thought it was like this:



-tiles  (table)

--- x  ( 0 , 0 is the same for all rmaps: a corner of the "google sphere" )

--- y

--- z  (17 - zoom_google, 17 is the earth on a single tile, 16 is the 4-tiles division ... )

--- s   (timestamp, can be set to 0 or anything ?)

--- image   (blob of the PNG)

- info (table)

--- minzoom ( for example 10 )

--- maxzoom  ( for example 17 if the maxzoom is the whole earth )



But I cant get OruxMaps to read some simple files with this structure:

" no compe" , "error leyendo mapa" and "error si coordinadas OK"

So what is the exact, and minimal, structure of RMAP open format, that Oruxmaps can parse ?

If it is close to this, it is interesting that mbtiles and rmaps are really close to each other. perhaps it would be possible to create compatible map files by adding some SQL "CREATE VIEW" to conform to each standard ?



thanks a lot !

yip

[In short: custom projection cannot be set for mbtiles maps.If forcibly set in the oruxmapsmaps.db internal database, it is resetted at the next mapsources rescan.]

Hi

In short: custom projection cannot be set for mbtiles maps.

If forcibly set in the oruxmapsmaps.db internal database,   it is resetted at the next mapsources rescan.

Is it possible to force a custom projection over a mbtiles ?

OK, official specs of the mbtiles format defines only one projection: Mercator Spherical ... or something like this.

So theorically, one would never ever need to define a custom datum/proj when using MBtiles.



With that said, mbtiles is becoming the de-facto standard for offline maps.



Still, some TMS don't serves maps in this projection. It is the case of Yandex and probably many others.

Therefore, when building an offline map from this service, mbtiles format cannot be used.



For some reason, I cannot build a reliable map from this service with anything else than mbtiles.

When importing a mbtiles generated from yandex, there is the known offset bug, due to Yandex using MercatorEllipsoidal projection instead of Mercator Spherical (documented in another thread).

Oruxmaps already supports Yandex for online sources via the XML config file.



Now, for offline use, the only option is to convert mbtiles to the otrk2 format, which doesn't seem easy to me.

Another option would be to convert mbtiles to one of the mysterious, undocumented "rmap" formats and attach a custom datum to it. (I spent dozens of hours trying to achieve this, to no avail, see next post).



Last resort, I ended up tinkering the internal oruxmaps database and made it work (sort of) with standard yandex mbtiles:

small workaround for anyone coming across this post, trying to use yandex offline maps, in mbtiles format:

*** WARNING : technical and geek stuff ahead ***

1. try to use Oruxmaps otrk2.xml format instead. no issue there. end.



2. import and load the mbtiles as usual (they will have the infamous offset for now)

3. Disable auto-reload of mapsource in settings

3. "root" your way to oruxmapsmaps.db and change PROJ from MERCATORESFERICA to MERCATORELIPSOIDAL with a sqlite editor

4. never rescan your mapsources or you will have to do step 3 again.

As always, orux, you provide an awesome app, I use it for a number of years, and totally love it.

This issue is not important, it is not very standard-compliant and probably isn't worth the time to investigate...

well it's for the record and info may be useful to others



yip