[File] [PATCH] Magdir/images, intel for Atari DEGAS bitmap *.pi1 *.pi2 *.pi3 *.pc1 *.pc2 *.pc3
Christos Zoulas
christos at zoulas.com
Fri Dec 10 18:29:22 UTC 2021
Committed, thanks!
christos
> On Dec 8, 2021, at 8:07 PM, Jörg Jenderek <joerg.jen.der.ek at gmx.net> wrote:
>
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Hello,
> some times ago i send patches for lif file because the starting 2
> byte magic 8000h was too weak. Unfortunately this starting sequence
> also occur for Atari DEGAS low-resolution bitmap (*.pc1).
> Now i inspect such Atari images with 6 different file name extension
> (pi1 pi2 pi3 pc1 pc2 pc3).
>
> When running running file command version 5.41 on such bitmaps and
> related files i get an output like:
> ARTIS3.PC2: data
> BEETHVEN.IMG: GEM Image data 224 x 131,
> 1 planes, 352 x 352 pixelsize
> CHURCH.IMG: GEM Image data 224 x 170,
> 1 planes, 352 x 352 pixelsize
> Faux-Spitzen.abr: data
> GAMEOVR4.IMG: GEM XIMG Image data 256 x 176,
> 4 planes, 372 x 372 pixelsize
> GNUCHESS.PC1: data
> LEREDACT.PI3: data
> SMTHDRAW.PC3: data
> TBX_DEMO.PI3: data
> bigspid.pi1: X11 SNF font data, MSB first
> bilboule.pi1: data
> clinton.img: GEM HYPERPAINT Image data 77 x 87,
> 4 planes, 338 x 372 pixelsize
> g3test.g3: data
> gnucash-4.8.setup.exe.aria2: data
> hpcc88.lif: lif file "HPCC88", version 1,
> directory length 12,
> extensions 0x4d00000002...,
> 1st file MCITFYP4
> load-v0001.aria2: data
> medres.pi2: data
> msvcrt.lib: Intel ia64 COFF object file,
> not stripped, 91 sections,
> symbol offset=0xa8d3, 644 symbols,
> created Tue Sep 19 05:18:08 2006,
> 1st section name ".drectve"
> plpbt.iso: ISO 9660 CD-ROM filesystem data
> 'Plop Boot Manager 5.0.14' (bootable)
> sigirl1.pi3: Intel ia64 COFF object file,
> not stripped, 256 sections,
> 1st section name ""
>
> For comparison reason i run the file format identification utility
> TrID ( See https://mark0.net/soft-trid-e.html).
> Most Atari Degas bitmap are described correctly by TrID as
> DEGAS bitmap by first two bytes, sometimes with low recognition rate.
> Only the PI1 examples are not recognised, probably because the
> patterns are too unspecific.
> The PI2 examples are described as "DEGAS med-res bitmap" by
> definition bitmap-pi2-degas.trid.xml. The PI3 examples are described
> as "DEGAS hi-res bitmap" by definition bitmap-pi3-degas.trid.xml.
> The PC1 examples are described as "DEGAS low-res compressed bitmap"
> by bitmap-pc1-degas.trid.xml. The PC2 examples are described as
> "DEGAS med-res compressed bitmap" by bitmap-pc2-degas.trid.xml.
> The PC3 examples are described as "DEGAS hi-res compressed bitmap"
> by bitmap-pc3-degas.trid.xml. (See appended DEGAS-trid-v.txt.gz ).
>
> Luckily TrID with -v option display correct file name extension and
> also URL pointing to file format specification. That is expressed
> inside Magdir/images by additional comment lines likes:
> # URL: http://fileformats.archiveteam.org/wiki/DEGAS_image
> # Reference: https://wiki.multimedia.cx/index.php?title=Degas
> # http://mark0.net/download/triddefs_xml.7z/defs/b
> # bitmap-pi2-degas.trid.xml bitmap-pi3-degas.trid.xml
> # bitmap-pc1-degas.trid.xml bitmap-pc2-degas.trid.xml
> # bitmap-pc3-degas.trid.xml
>
> On that site download links for examples and graphic tools are
> mentioned. I verified information by NetPBM tool pi3topbm, XnView
> command line tool nconvert and Deark software.
>
> Unfortunately there exist no strong significant magic pattern (only 2
> bytes) for such bitmap images. So i put displaying part inside sub
> routine degas-bitmap which starts like:
> 0 name degas-bitmap
>> 0 ubyte x Atari DEGAS
> !:mime image/x-atari-degas
>> 0 ubyte =0x80 Elite compressed
>>> 32042 ubequad x Elite
>> 0 beshort 0x0000 bitmap
> !:ext pi1
>> 0 beshort 0x0001 bitmap
> !:ext pi2
>> 0 beshort 0x0002 bitmap
> !:ext pi3
>> 0 beshort 0x8000 bitmap
> !:ext pc1
>> 0 beshort 0x8001 bitmap
> !:ext pc2
>> 0 beshort 0x8002 bitmap
> !:ext pc3
>> 1 ubyte =0 320 x 200 x 16
>> 1 ubyte =1 640 x 200 x 4
>> 1 ubyte =2 640 x 400 x 2
>
> The first byte determinate if the image bytes are compressed. The
> value 80h means compressed variant and 0 means uncompressed.
> The second byte determinate the image resolution and colour depth.
> The value 0 means low resolution (320×200, 16 colours). One means
> medium resolution (640×200, 4 colours) and value two means high
> resolution (640×400, 2 colours). Depending on compression and
> resolution the file name extension varies. The documentation also
> mention a second extension SUH for uncompressed high resolution,
> but i myself found no such examples. Instead of generic mime type
> application/octet-stream i show a user defined one. For the
> uncompressed images exist a "elite" variant. There after the pixel
> data animation information about colour to change, direction and
> time delay is stored. This can also be shown at the end by
> additional debugging lines like:
>> 32034 ubequad !0 \b, color animations %16.16llx (left)
>>> 32042 ubequad !0 %16.16llx (right)
>> 32050 ubequad !0 \b, channel directions %16.16llx
>> 32058 ubequad !0 \b, channel delays %16.16llx
>
> After the 2 starting bytes the colour palette is stored til offset
> 34. The Atari ST palette has place for 16 colour entries. Each entry
> occupies 2 bytes ( or 16 bits) in big endian order. Not all 16 bits
> are used. Some examples use only 9 bits ( ?????RRR?GGG?BBB with R for
> red, G for green, B for Blue and ? for unused. That means 512
> different colours) and some use 12 bits ( ????RRRRGGGGBBBB . That
> means 4096 different colours). In documentation also an Atari
> Spectrum 512 Enhanced 15 bit palette is mentioned, but luckily this
> variant apparently does not be used for DEGAS images. According to
> documentation you can not rely that unused bits are zero like in
> mentioned "bad" example bilboule.pi1, but i myself found that in most
> examples unused bits are zero ( so replace ? by 0 in bit mask).
> For control reason show the first colour entries by lines like:
>> 2 ubeshort x \b, color palette %4.4x
>> 4 ubeshort x %4.4x
>> 6 ubeshort x %4.4x
>> 8 ubeshort x %4.4x
>> 10 ubeshort x %4.4x
> So often we see value 0000. That means black colour. The value 0777
> means white colour in 9 bit variant and 0fff means white in 12 varian
> t.
> So this information can be used as an additional test. So DEGAS
> low-resolution compressed bitmaps (like: BATTLSHP.PC1 GNUCHESS.PC1
> MEDUSABL.PC1 MOONLORD.PC1 WILDROSE.PC1) are recognized by lines like:
> 0 beshort 0x8000
>> 2 ubeshort&0xF000 0
>>> 0 use degas-bitmap
>
> The first test for starting two bytes is also true for lif files like
> hpcc88.lif handled by Magdir/lif. But there at offset 2 the volume
> label ( ASCII like HPCC88 or hexadecimal 485043433838 in example
> hpcc88.lif) is stored. So by second test line for unused (that means
> zero) colour palette bits lif examples are skipped and the test
> succeeds for PC1 bitmaps.
> PI1 examples like bigspid.pi1 and bilboule.pi1 are done by lines like
> :
> 0 beshort 0x0000
>> 2 uquad !0
>>> 4 ubeshort&0xF000 0
>>>> 0 use degas-bitmap
> Here by second test line some "bad" ISO 9660 CD-ROM filesystems like
> plpbt.iso are skipped. Zero value here would be interpreted as 4
> black colours at the beginning of the colour palette. For real DEGAS
> bitmaps one could find one black colour (that means value 0000) in
> the colour palette, but then the other entries have other colours
> (non zero values) in other entries. So example plpbt.iso with
> 8-byte zero value is skipped.
>
> For Atari mid-res DEGAS bitmap PI2 examples the lines looks like
> 0 beshort 0x0001
> ...
>>>>>>>> 32026 quad x
>>>>>>>>> 0 use degas-bitmap
> After test for weak 2 starting bytes by test line eight the GEM
> HYPERPAINT Image clinton.img is skipped by check for existence of
> bytes at the end of DEGAS images.
>
> The differentiation between DEGAS PI3 examples and Adobe PhotoShop
> Brush ABR is a little bit tricky. For debugging purpose show my
> observed information about ABR by lines like:
>>> 19 ubyte !0 \b, NOTE LENGTH %u
>>>> 21 lestring16 x \b, BRUSH NOTE "%s"
> So for example Faux-Spitzen.abr i get note string "Gitter - klein "
> with length 15. So for example "Verschiedene Spitzen.abr" i get with
> length 8 the note string "Kreis 1 ".
>
> So if this string length is zero i assume that is not an ABR. So such
> examples must be DEGAS images. So many examples ( like: 4th_ofj2.pi3
> GEMINI03.PI3 PEOPLE18.PI3 POWERFIX.PI3 abydos.pi3 highres.pi3
> sigirl1.pi3 vanna5.pi3) are handled by branch like:
>>> 19 ubyte =0
>>>> 0 use degas-bitmap
> Because brush note string is stored as UTF-16 the stored string
> length multiplied with 2 gives the number of bytes occupies by that
> string. So this information can be used to inspect the last
> last character of Adobe PhotoShop Brush UTF16-LE string and
> terminating nil character (that are 4 bytes) by lines like
>>> 19 ubyte !0
>>>> (19.b*2) ubequad x
>>>>> &8 ubelong x \b, LAST CHAR+NIL %8.8x
> For example "Faux-Spitzen.abr" here i get hexadecimal value 006e0000
> (character n) and for "Verschiedene Spitzen.abr" i get hexadecimal
> value 00310000 (character 1).
>
> So when test for such nil bytes gives non zero value, it must be
> DEGAS image. So many PI3 examples (like ARABDEMO.PI3 ELMRSESN.PI3
> GEMVIEW.PI3 LEREDACT.PI3 PICCOLO.PI3 REPRO_JR.PI3 ST_TOOLS.PI3
> TBX_DEMO.PI3 evgem7.pi3) are skipped by other branch with additional
> nil test lines like:
>>>>> &8 ubelong&0xff00ffFF !0
>>>>>> 0 use degas-bitmap
>
> If test for such nil bytes gives zero value, test it again but now
> looks if place of last character of note string contains value is
> valid. If this value is "too low" (that means non printable
> character) it must be a DEGAS image. If this value is "high" enough
> is is a "normal" printable character. That means it is a Photoshop
> ABR. By this last branch the remaining last DEGAS bitmaps ( like
> BASICNES.PI3 DB_HELP.PI3 DB_WRITR.PI3 LEREDACT.PI3) are skipped by
> lines like:
>>>>> &8 ubelong&0xff00ffFF =0
>>>>>> &-4 ubelong&0x00FF0000 <0x00200000
>>>>>>> 0 use degas-bitmap
>
> Some DEGAS high-res uncompressed bitmaps (like GEMINI03.PI3
> MODEM2.PI3 POWERFIX.PI3 sigirl1.pi3 vanna5.pi3) are misidentified as
> "Intel ia64 COFF object file" because the 2 byte start pattern is the
> same. That was expressed by lines inside Magdir/intel like:
> 0 leshort 0x0200
>> 0 use display-coff
> Luckily the displaying part is done by subroutine display-coff inside
> Magdir/coff. So only additional test lines must be inserted before
> calling sub routine. For all my misidentified PI3 examples the
> interpreted first section name was nil, whereas for real COFF object
> file we get here typical 8-byte sized names (like .text .data
> .debug$S .drectve .testseg). So for most COFF objects the starting
> character is a point character (0x2E). If i remember right some
> Borland compiler for example use DATA instead of .data. So at least
> i assume that starting character is like ASCII printable (that
> means value "high enough"), whereas for DEGAS image at that offset
> the colour palette entry number 10 starts. Because of 4 reserved
> bits (that are in most cases zero) we get here a low value.
> So DEGAS images can be distinguished with very high rate from COFF
> object files. Unfortunately the section name can appear later if COFF
> sample contains an optional header. But in documentation is written
> that COFF object files have not header part compared with COFF
> executables. So i must check by additional second test for F_EXEC
> flag bit. If flag is set, then it is an executable and i can
> call directly sub routine. If this bit is not set, is is an object
> file and i check for starting character of section name befor calling
> sub routine. So magic lines now become like:
> 0 leshort 0x0200
>> 18 leshort ^0x0002
>>> 20 ubyte >0x1F
>>>> 0 use display-coff
>> 18 leshort &0x0002
>>> 0 use display-coff
>
> After applying the above mentioned modifications by patches
> file-5.41-images-degas.diff and file-5.41-intel-pi3.diff
> then all my Degas bitmaps are correctly identified and some
> misidetfication vanish like:
>
> ARTIS3.PC2: Atari DEGAS Elite compressed bitmap
> 640 x 200 x 4, color palette
> 0fff 0f00 00f0 0000 0007 ...
> BEETHVEN.IMG: GEM Image data 224 x 131,
> 1 planes, 352 x 352 pixelsize
> CHURCH.IMG: GEM Image data 224 x 170,
> 1 planes, 352 x 352 pixelsize
> Faux-Spitzen.abr: data
> GAMEOVR4.IMG: GEM XIMG Image data 256 x 176,
> 4 planes, 372 x 372 pixelsize
> GNUCHESS.PC1: Atari DEGAS Elite compressed bitmap
> 320 x 200 x 16, color palette
> 0221 0000 0310 0420 0530 ...
> LEREDACT.PI3: Atari DEGAS Elite bitmap
> 640 x 400 x 2, color palette
> 0fff 0f00 00f0 0000 0fff ...
> SMTHDRAW.PC3: Atari DEGAS Elite compressed bitmap
> 640 x 400 x 2, color palette
> 0777 0700 0070 0000 0777 ...
> TBX_DEMO.PI3: Atari DEGAS Elite bitmap
> 640 x 400 x 2, color palette
> 0777 0700 0070 0000 0777 ...
> bigspid.pi1: Atari DEGAS Elite bitmap
> 320 x 200 x 16, color palette
> 0004 0025 0037 0000 0410 ...
> bilboule.pi1: Atari DEGAS Elite bitmap
> 320 x 200 x 16, color palette
> 0000 0111 8222 0333 fcc4 ...
> clinton.img: GEM HYPERPAINT Image data 77 x 87,
> 4 planes, 338 x 372 pixelsize
> g3test.g3: data
> gnucash-4.8.setup.exe.aria2: data
> hpcc88.lif: lif file "HPCC88", version 1,
> directory length 12,
> extensions 0x4d00000002...,
> 1st file MCITFYP4
> load-v0001.aria2: data
> medres.pi2: Atari DEGAS Elite bitmap
> 640 x 200 x 4, color palette
> 0777 0700 0070 0000 0007 ...
> msvcrt.lib: Intel ia64 COFF object file,
> not stripped, 91 sections,
> symbol offset=0xa8d3, 644 symbols,
> created Tue Sep 19 05:18:08 2006,
> 1st section name ".drectve"
> plpbt.iso: ISO 9660 CD-ROM filesystem data
> 'Plop Boot Manager 5.0.14' (bootable)
> sigirl1.pi3: Atari DEGAS Elite bitmap
> 640 x 400 x 2, color palette
> 0001 0000 0000 0000 0000 ...
>
> I hope my diff files can be applied in future version of file utility
> .
>
>
> With best wishes
> Jörg Jenderek
> - --
> Jörg Jenderek
>
>
> -----BEGIN PGP SIGNATURE-----
> Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/
>
> iF0EARECAB0WIQS5/qNWKD4ASGOJGL+v8rHJQhrU1gUCYbFWxwAKCRCv8rHJQhrU
> 1rqHAKCpHT9jL3dmki5XL/+jSjpzVHjyYwCfXx5NiYn9nsJ3jwLeoDMKgJCbPqg=
> =Wm0D
> -----END PGP SIGNATURE-----
> <file-5_41-images-degas_diff.DEFANGED-2231><file-5_41-images-degas_diff_sig.DEFANGED-2232><DEGAS-trid-v.txt.gz><file-5_41-intel-pi3_diff.DEFANGED-2233><file-5_41-intel-pi3_diff_sig.DEFANGED-2234>--
> File mailing list
> File at astron.com
> https://mailman.astron.com/mailman/listinfo/file
> <sanitizer.log>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 235 bytes
Desc: Message signed with OpenPGP
URL: <https://mailman.astron.com/pipermail/file/attachments/20211210/6feaea2a/attachment.asc>
More information about the File
mailing list