[mkgmap-dev] Text format for TYP files

[Marko Mäkelä]

This may be slightly off-topic, I think that it could be useful to have
a repository of TYP files and associated style definitions. For example,
a TYP file would be necessary to define a custom line style for displaying
bus route information on a separate map layer.

It seems that http://cypherman1.googlepages.com/genTYP is the "de facto"
text format of TYP files. According to the home page, the latest release
was about 3 years ago.

The web based TYP file editor at http://ati.land.cz/gps/typdecomp/
is more recent, but its text output does not seem to be able to export
all the definitions that the editor is able to create. Some of the
source code is available, but there are very few comments (some in Chech)
and no example of usage. I was not enough of a Perl expert to figure out
how to input the binary TYP data for decompilation. For what it is worth,
I tried the following, but my call signature of the as_string method is
obviously wrong.

#!/usr/bin/perl
require "source.cgi";
undef $/; $_=<>; print Garmin::TYP::as_string($_);

I wonder if there are any other tools that make use of the genTYP format.
A textual format like genTYP's (with XPM graphics) would be ideal for a
TYP file repository. It would be easy to change parameters
(such as Family ID) and copy definitions between TYP files with any
text editor.

Any thoughts? Would it be worthwhile to reimplement the genTYP txt/TYP
encoder/decoder in Java? It should not be too hard, given that an XPM
library already exists at <http://www.bolthole.com/java/Xpm.html>.

Marko

[Felix Hartmann]

Post by Marko Mäkelä
This may be slightly off-topic, I think that it could be useful to have
a repository of TYP files and associated style definitions. For example,
a TYP file would be necessary to define a custom line style for displaying
bus route information on a separate map layer.
It seems that http://cypherman1.googlepages.com/genTYP is the "de facto"
text format of TYP files. According to the home page, the latest release
was about 3 years ago.
The web based TYP file editor at http://ati.land.cz/gps/typdecomp/
is more recent, but its text output does not seem to be able to export
all the definitions that the editor is able to create. Some of the
source code is available, but there are very few comments (some in Chech)
and no example of usage. I was not enough of a Perl expert to figure out
how to input the binary TYP data for decompilation. For what it is worth,
I tried the following, but my call signature of the as_string method is
obviously wrong.
#!/usr/bin/perl
require "source.cgi";
undef $/; $_=<>; print Garmin::TYP::as_string($_);
I wonder if there are any other tools that make use of the genTYP format.
A textual format like genTYP's (with XPM graphics) would be ideal for a
TYP file repository. It would be easy to change parameters
(such as Family ID) and copy definitions between TYP files with any
text editor.

I use maptk: http://maptk.dnsalias.com/
it is freeware but not OS - it is however available for Python, Linux
and Windows.

This is the most reliable TYP-File editor/compilor for me. It supports
in text editing and also exporting to genTYP format (PRJ to mapedit
skin). It does not support GUI night mode editing however (I think one
can edit nightmode in text editor).

http://ati.land.cz/gps/typdecomp/

has some problems with newer TYP files as used with gmap format maps. I
also find it really slow to use.

Post by Marko Mäkelä
Any thoughts? Would it be worthwhile to reimplement the genTYP txt/TYP
encoder/decoder in Java? It should not be too hard, given that an XPM
library already exists at<http://www.bolthole.com/java/Xpm.html>.
Marko
_______________________________________________
mkgmap-dev mailing list
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

[Marko Mäkelä]

Hi Felix, all,

Post by Felix Hartmann
I use maptk: http://maptk.dnsalias.com/
it is freeware but not OS - it is however available for Python, Linux
and Windows.

Thank you for the hint. I downloaded "MapTk 2.7 Python.zip". The software
is in .pyc files that have apparently been compiled on Windows. The only
non-binary file is MapTk.pyw, which requires python-tk and Tcl/Tk to run.
I decompiled and compiled one simple TYP file (two line styles) that I had
created with http://ati.land.cz/gps/typdecomp/. It was 2 bytes shorter
than before, even though the string "MapTk 2.7\0" was appended to the end.
That is, some things were possibly lost in the conversion. It could be
harmless, such as representing the lines with fewer bits per pixel
(I used 2 colors even though it was declared as 4-colour).

Post by Felix Hartmann
This is the most reliable TYP-File editor/compilor for me. It supports
in text editing and also exporting to genTYP format (PRJ to mapedit
skin). It does not support GUI night mode editing however (I think one
can edit nightmode in text editor).

The .prj file that was decompiled from the TYP file follows the Windows .ini
style. The "PRJ file to MapEdit skin" produces something genTYP-like, but
it lacks some information, such as Product ID and Family ID.

I do not mind using a non-free TYP editor for now, but I would like to use
a text format that can represent every aspect of the TYP file. Later, if
a better TYP editor is released as free software, we could convert the source
files to the format of that editor if needed.

I think that we could do with a simple text-to-TYP compiler. I would
implement it in Java, because users are likely to have the runtime
environment installed already, for splitter, mkgmap and JOSM.

For regression testing, we could have a set of TYP source files that would
contain the MD5 checksum of the resulting binary in a comment.

What do you think? How are you storing your TYP files, Felix? In the
MapTk PRJ file format, or as binary files?

Marko

[Felix Hartmann]

Post by Marko Mäkelä
Hi Felix, all,

Post by Felix Hartmann
I use maptk: http://maptk.dnsalias.com/
it is freeware but not OS - it is however available for Python, Linux
and Windows.

Thank you for the hint. I downloaded "MapTk 2.7 Python.zip". The software
is in .pyc files that have apparently been compiled on Windows. The only
non-binary file is MapTk.pyw, which requires python-tk and Tcl/Tk to run.
I decompiled and compiled one simple TYP file (two line styles) that I had
created with http://ati.land.cz/gps/typdecomp/. It was 2 bytes shorter
than before, even though the string "MapTk 2.7\0" was appended to the end.
That is, some things were possibly lost in the conversion. It could be
harmless, such as representing the lines with fewer bits per pixel
(I used 2 colors even though it was declared as 4-colour).

maptk only supports 2 colors (as no nightmode support is included, 4
colors mean 2 colors daymode and 2 colors in nightmode). It could well
be that maptk did some changes to the colour palette. I think it tries
to make colors work more or less the same on old 256 colour units (like
legen/vista/60CSx/edge) as for 16/24bit displays.

Post by Marko Mäkelä

Post by Felix Hartmann
This is the most reliable TYP-File editor/compilor for me. It supports
in text editing and also exporting to genTYP format (PRJ to mapedit
skin). It does not support GUI night mode editing however (I think one
can edit nightmode in text editor).

The .prj file that was decompiled from the TYP file follows the Windows .ini
style. The "PRJ file to MapEdit skin" produces something genTYP-like, but
it lacks some information, such as Product ID and Family ID.
I do not mind using a non-free TYP editor for now, but I would like to use
a text format that can represent every aspect of the TYP file. Later, if
a better TYP editor is released as free software, we could convert the source
files to the format of that editor if needed.
I think that we could do with a simple text-to-TYP compiler. I would
implement it in Java, because users are likely to have the runtime
environment installed already, for splitter, mkgmap and JOSM.
For regression testing, we could have a set of TYP source files that would
contain the MD5 checksum of the resulting binary in a comment.
What do you think? How are you storing your TYP files, Felix? In the
MapTk PRJ file format, or as binary files?

I'm working from .prj MapTk files. It's the quickest way if you want to
update several typfiles at the same time. I even think about moving to
track or svn monitoring for the typfiles so that I only need to do my
changes in one file.
I once or twice hat to go to backups after using ati.land as it
destroyed my typfiles, and also like the maptk GUI editor (the only
major problem is that is lacks a png import function). I also find the
.prj format easier to understand/edit in texteditor in comparison to
cgpsmapper/gentyp notation.

I never used night mode, but for in car use it is useful (or better said
for non "topographic" maps that have few polygons but thick streets). I
would support a text notation that one can at least for roads and
polygons easily imagine - with genTYP format I do have problems
imagining how it will look like once compiled.

Post by Marko Mäkelä
Marko
_______________________________________________
mkgmap-dev mailing list
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

[Marko Mäkelä]

Post by Felix Hartmann
maptk only supports 2 colors (as no nightmode support is included, 4
colors mean 2 colors daymode and 2 colors in nightmode). It could well
be that maptk did some changes to the colour palette. I think it tries
to make colors work more or less the same on old 256 colour units (like
legen/vista/60CSx/edge) as for 16/24bit displays.

OK, I see. It should be good enough for simple TYP files then.

Post by Felix Hartmann
I'm working from .prj MapTk files. It's the quickest way if you want to
update several typfiles at the same time. I even think about moving to
track or svn monitoring for the typfiles so that I only need to do my
changes in one file.

I try to keep all my important files in svn, except write-once files, such
as photos, GPS track logs and email (Mutt's maildir format is one file per
message). You don't need a server process for that; you can use the file:///
URL for a local repository. The FSFS repository format is nice.

Post by Felix Hartmann
I once or twice hat to go to backups after using ati.land as it
destroyed my typfiles, and also like the maptk GUI editor (the only
major problem is that is lacks a png import function). I also find the
.prj format easier to understand/edit in texteditor in comparison to
cgpsmapper/gentyp notation.

Do you know the author of maptk? Could he be convinced to release the
source code of the TYP compiler under the GPL? If not, how much and how
often has the PRJ file format been changed in the past? In other words,
would it be reasonable to write an independent PRJ to TYP compiler?

Post by Felix Hartmann
I never used night mode, but for in car use it is useful (or better said
for non "topographic" maps that have few polygons but thick streets). I
would support a text notation that one can at least for roads and
polygons easily imagine - with genTYP format I do have problems
imagining how it will look like once compiled.

The test file that I created with the ati.land.cz editor did define
separate day/night colors for a line label. The decompiler did not
fully know about this:

; additional block with colors:
; 00000000 1A 00 00 CC 40 40 80 ....@@.

In the [Polyline] section of the MapTK PRJ file, only the day label color
is included:

TextColor=0xcc0000

I used automatic day/night mode on my Edge 705 (bicycle) for a several months,
but in the end I switched to day mode only, because thin brown paths
(cycleways) are difficult to read on solid black background and almost
unreadable on black/gray forest areas. The night mode might make more
sense on a CRT or OLED screen than on a low-contrast backlit LCD anyway.
On a backlit display, you could as well set the brightness to minimum.

So, I think that the "repository of simple map layer styles and TYP files"
could do without the night mode at start.

Marko

[Felix Hartmann]

Post by Marko Mäkelä

Post by Felix Hartmann
maptk only supports 2 colors (as no nightmode support is included, 4
colors mean 2 colors daymode and 2 colors in nightmode). It could well
be that maptk did some changes to the colour palette. I think it tries
to make colors work more or less the same on old 256 colour units (like
legen/vista/60CSx/edge) as for 16/24bit displays.

OK, I see. It should be good enough for simple TYP files then.

Post by Felix Hartmann
I'm working from .prj MapTk files. It's the quickest way if you want to
update several typfiles at the same time. I even think about moving to
track or svn monitoring for the typfiles so that I only need to do my
changes in one file.

I try to keep all my important files in svn, except write-once files, such
as photos, GPS track logs and email (Mutt's maildir format is one file per
message). You don't need a server process for that; you can use the file:///
URL for a local repository. The FSFS repository format is nice.

Post by Felix Hartmann
I once or twice hat to go to backups after using ati.land as it
destroyed my typfiles, and also like the maptk GUI editor (the only
major problem is that is lacks a png import function). I also find the
.prj format easier to understand/edit in texteditor in comparison to
cgpsmapper/gentyp notation.

Do you know the author of maptk? Could he be convinced to release the
source code of the TYP compiler under the GPL? If not, how much and how
often has the PRJ file format been changed in the past? In other words,
would it be reasonable to write an independent PRJ to TYP compiler?

I know him a bit from naviuser.at and naviboard.de online forums. The
PRJ format seems to have been consistent since I can remember. The GUI
editor has been changing a bit in the past however (especially in the
regard to what it finds correct and incorrect, so some things before
being corrected had to be written in text for implementation). I have no
clue whether he would be willing to release that part of code under GPL.

If you like I could forward (send him a link) our conversation and we
could see what he responds.

Post by Marko Mäkelä

Post by Felix Hartmann
I never used night mode, but for in car use it is useful (or better said
for non "topographic" maps that have few polygons but thick streets). I
would support a text notation that one can at least for roads and
polygons easily imagine - with genTYP format I do have problems
imagining how it will look like once compiled.

The test file that I created with the ati.land.cz editor did define
separate day/night colors for a line label. The decompiler did not
In the [Polyline] section of the MapTK PRJ file, only the day label color
TextColor=0xcc0000
I used automatic day/night mode on my Edge 705 (bicycle) for a several months,
but in the end I switched to day mode only, because thin brown paths
(cycleways) are difficult to read on solid black background and almost
unreadable on black/gray forest areas. The night mode might make more
sense on a CRT or OLED screen than on a low-contrast backlit LCD anyway.
On a backlit display, you could as well set the brightness to minimum.
So, I think that the "repository of simple map layer styles and TYP files"
could do without the night mode at start.
Marko
_______________________________________________
mkgmap-dev mailing list
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

[Marko Mäkelä]

Post by Felix Hartmann

Post by Marko Mäkelä
Do you know the author of maptk? Could he be convinced to release the
source code of the TYP compiler under the GPL? If not, how much and how
often has the PRJ file format been changed in the past? In other words,
would it be reasonable to write an independent PRJ to TYP compiler?

I know him a bit from naviuser.at and naviboard.de online forums. The
PRJ format seems to have been consistent since I can remember. The GUI
editor has been changing a bit in the past however (especially in the
regard to what it finds correct and incorrect, so some things before
being corrected had to be written in text for implementation). I have no
clue whether he would be willing to release that part of code under GPL.
If you like I could forward (send him a link) our conversation and we
could see what he responds.

We could do that, but I just got another idea: I will create a simple
TYP file with http://ati.land.cz/gps/typdecomp/ and try to decompile
it by hand into source file(s) that would be compiled with GNU Assembler
to the TYP file. That should be enough for my immediate needs, to define
some line styles. I will probably need a couple of weeks for that.

POI icons could be converted to a suitable format by a script based on
NetPBM (or ImageMagick or some other graphics conversion toolset) and od.

An important consequence of this approach would be that there would be no
GUI for editing the TYP files (except a graphics editor for the POI icons).
A GUI-driven code generator could of course be implemented later.

Another important consequence would be that the source file(s) would
explicitly define each and every byte in the output file. The complexity
would be hidden by defining and using appropriate macros and symbols.

Marko

[Greg Troxel]

We could do that, but I just got another idea: I will create a simple
TYP file with http://ati.land.cz/gps/typdecomp/ and try to decompile
it by hand into source file(s) that would be compiled with GNU Assembler
to the TYP file. That should be enough for my immediate needs, to define
some line styles. I will probably need a couple of weeks for that.

That might work, if the format is regular enough. You're writing the
code, but my first inclination is to define xml for the TYP file and
write something that uses an xml parser to an in-memory structure and
the write out the bits.

[Felix Hartmann]

Post by Marko Mäkelä
We could do that, but I just got another idea: I will create a simple
TYP file with http://ati.land.cz/gps/typdecomp/ and try to decompile
it by hand into source file(s) that would be compiled with GNU Assembler
to the TYP file. That should be enough for my immediate needs, to define
some line styles. I will probably need a couple of weeks for that.
That might work, if the format is regular enough. You're writing the
code, but my first inclination is to define xml for the TYP file and
write something that uses an xml parser to an in-memory structure and
the write out the bits.

O I just remember the biggest problem with ati.land. It does drop some
"3byte / extended" types. Be careful and look at the sourcecode if you
want to reverse engineer.

Post by Marko Mäkelä
_______________________________________________
mkgmap-dev mailing list
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

[Marko Mäkelä]

Hi Greg,

Post by Greg Troxel

Post by Marko Mäkelä
We could do that, but I just got another idea: I will create a simple
TYP file with http://ati.land.cz/gps/typdecomp/ and try to decompile
it by hand into source file(s) that would be compiled with GNU Assembler
to the TYP file. That should be enough for my immediate needs, to define
some line styles. I will probably need a couple of weeks for that.

That might work, if the format is regular enough. You're writing the
code, but my first inclination is to define xml for the TYP file and
write something that uses an xml parser to an in-memory structure and
the write out the bits.

That is of course a possibility. Using XML and Java would have the benefit
that mkgmap is are already using them.

Would XSLT be able to cut the mustard? According to
http://www.oreillynet.com/xml/blog/2008/06/xslt_and_binary_file_formats_1.html
it might be.

I would rather choose a well-known tool that can represent the TYP file
with minimum hassle. The assembler could be replaced with something
else later. Well, you will see once I have some code to present.

Post by Greg Troxel
That compiler is now under a BSD license, so I wonder about just making
it into a regular program instead of a cgi.

Have you tried using the source code? I tried, but failed. It does not
even claim to be the entire program. The web interface is deliberately
missing. The code seems to implement a transformation from in-memory
data structures to textual and binary TYP formats, in the methods as_string
and as_binary. The constructors seem to be taking the binary TYP in the
$raw_data parameter. I have done some Perl programming in the past, but
I have never written object-oriented Perl. It would be easier to work with
the code if there were more comments or if I knew some Chech.

Best regards,

Marko

[Greg Troxel]

Have you tried using the source code? I tried, but failed. It does not
even claim to be the entire program. The web interface is deliberately

No, I have not tried. That doesn't sound encouraging.

[Clinton Gladstone]

Post by Marko Mäkelä
Have you tried using the source code?

I did take a look at this some time ago. I noted that there were a lot of dependencies (such as imagemagick) which could cause troubles. Instead I used the source code to help me understand the TYP file format: I used this knowledge to write a small Perl script which changes the Family and Product IDs of a given TYP file.

Based on this experience, it may be advisable for you to also create your program "from scratch", using the http://ati.land.cz/ program as a reference.

Cheers.

[Carlos Dávila]

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=ISO-8859-1" http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
Clinton Gladstone wrote:
<blockquote
cite="mid:D0CD408B-6795-4925-BE27-***@googlemail.com"
type="cite">
<pre wrap=""><!---->
I did take a look at this some time ago. I noted that there were a lot of dependencies (such as imagemagick) which could cause troubles. Instead I used the source code to help me understand the TYP file format: I used this knowledge to write a small Perl script which changes the Family and Product IDs of a given TYP file.
</pre>
</blockquote>
Would you be so kind to share your script with this list? Having to
maintain several TYP files that only differ in FID and PID is an
annoyance using the on line editor.<br>
Thanks in advance.<br>
Carlos<br>
</body>
</html>

[Clinton Gladstone]

Post by Clinton Gladstone
I used this knowledge to write a small Perl script which changes the Family and Product IDs of a given TYP file.

Would you be so kind to share your script with this list? Having to maintain several TYP files that only differ in FID and PID is an annoyance using the on line editor.

Sorry for not replying earlier: I had to recover my system from a hard drive failure.

I have attached the script. It should be fairly straight forward to use:

set-typ.pl <family id> <product id> <TYP file>

I would recommend always running this on a copy of your TYP file, just in case something goes wrong. :-)

Enjoy.

[Carlos Dávila]

Post by Clinton Gladstone

Post by Clinton Gladstone
I used this knowledge to write a small Perl script which changes the Family and Product IDs of a given TYP file.

Would you be so kind to share your script with this list? Having to maintain several TYP files that only differ in FID and PID is an annoyance using the on line editor.

Sorry for not replying earlier: I had to recover my system from a hard drive failure.
set-typ.pl <family id> <product id> <TYP file>
I would recommend always running this on a copy of your TYP file, just in case something goes wrong. :-)
Enjoy.

Thank you very much. It works great!

Post by Clinton Gladstone
------------------------------------------------------------------------
_______________________________________________
mkgmap-dev mailing list
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

[Steve Ratcliffe]

Hi

There is code for generating TYP files under the ...imgfmt.app.typ
package, and I may have some more code from Thomas that what is checked
in which I could send you if you want to follow that route.

..Steve

[Marko Mäkelä]

Hi Steve,

Post by Steve Ratcliffe
There is code for generating TYP files under the ...imgfmt.app.typ
package, and I may have some more code from Thomas that what is checked
in which I could send you if you want to follow that route.

Thanks, it could be a good long-term solution.

In any case, I will first have to see how ugly the assembler solution
would become. At least it would be extremely flexible for prototyping.

Marko

[Greg Troxel]

That compiler is now under a BSD license, so I wonder about just making
it into a regular program instead of a cgi.