[File] [PATCH] Magdir/archive POSIX tar generic for Webmin Module, Theme
Jörg Jenderek (GMX)
joerg.jen.der.ek at gmx.net
Mon Jul 22 21:37:58 UTC 2024
Hello,
i was not satisfied with GUI configuration tools on my Raspian systems.
So i looked for alternatives. So i try webmin. The functionality is
expandable by modules. These typically have file name suffix WBM.
Some samples have file name suffix WBT. This means Webmin Theme.
When i run newest file command on such samples and related files with -e
tar option i get an output like:
FRITZ.Box_7520_B-07.57.image: AVM FRITZ!Box firmware HW276
(FRITZ!Box 7520 B),
version 07.57, directory
./var/, mode 0040755, uid
0000000, gid 0000000, size 00000000000,
seconds 14474476653, file
./var/content, mode 0100644, uid
0000000, gid 0000000, size 00000000543,
seconds 14474476653
apcupsd-0.81-2.wbm: POSIX tar archive (GNU), directory
apcupsd/, mode 0040700, uid
0000000, gid 0000001, size 00000000000,
seconds 07172756000, user
root, group bin
authentic-theme-21.09.5.wbt: POSIX tar archive (GNU), directory
authentic-theme/, mode 0000755, uid
0000765, gid 0000024, size 00000000000,
seconds 14522234321, user
user, group staff
cwmail.wbm: POSIX tar archive, directory
cwmail, mode 000755 , uid
000016 , gid 000004 , size 00000000000 ,
seconds 07102263376, user
grund, group bin, devmin 476230
etcupdate.wbm: POSIX tar archive (GNU), directory
etcupdate/, mode 0000755, uid
0000000, gid 0000000, size 00000000000,
seconds 10300072753, user
root, group root
link.wbm: POSIX tar archive (GNU), directory
link/, mode 0040775, uid
0005671, gid 0000012, size 00000000000,
seconds 07627514126, user
jcameron, group wheel
lpadmin.wbm: POSIX tar archive (GNU), directory
lpadmin/, mode 0000755, uid
0005671, gid 0000012, size 00000000000,
seconds 14513515572, user
jcameron, group wheel
memcached.wbm: POSIX tar archive (GNU), directory
memcached/, mode 0000755, uid
0001750, gid 0000144, size 00000000000,
seconds 11671622453, user
user, group users
old-blue-theme-1.0.wbm: POSIX tar archive (GNU), directory
old-blue-theme/, mode 0000755, uid
0005671, gid 0000012, size 00000000000,
seconds 12457303450, user
jcameron, group wheel
old-blue-theme.wbm: POSIX tar archive (GNU), directory
old-blue-theme/, mode 0040755, uid
0005671, gid 0000012, size 00000000000,
seconds 10533114330, user
jcameron, group wheel
old-mscstyle3.wbm: POSIX tar archive (GNU), directory
old-mscstyle3/, mode 0040755, uid
0005671, gid 0000000, size 00000000000,
seconds 10343426200, user
jcameron, group root
virtual-server-mobile.wbm: POSIX tar archive (GNU), directory
virtual-server-mobile/, mode 0040775, uid
0005671, gid 0000012, size 00000000000,
seconds 11047144366, user
jcameron, group wheel
virtual-server-theme-3.4.wbt: POSIX tar archive (GNU), directory
virtual-server-theme/, mode 0040775, uid
0005671, gid 0000012, size 00000000000,
seconds 10562743567, user
jcameron, group wheel
webmin.wbm: POSIX tar archive (GNU), directory
webmin/, mode 0000755, uid
0005671, gid 0000012, size 00000000000,
seconds 14513515571, user
jcameron, group wheel
With option --extension for samples wrong tar/gtar or tar/ustar suffix
are shown. With -i option application/x-gtar or application/x-ustar is
shown.
For comparison reason i also run the file format identification utility
DROID (See https://sourceforge.net/projects/droid/). Here the samples
are recognized generic. These are described as "Tape Archive Format"with
mime type application/x-tar by PUID x-fmt/265.
On Linux according to shared MIME-info database such samples are called
"Tar archive". Here application/x-gtar is used as mime type. The
samples are just recognized by looking for 8 byte sequence
ustar\040\040\0 at offset 257. Here 3 suffix (tar gtar gem) are listed.
That information can be seen in source freedesktop.org.xml.in found for
example on gitlab.freedesktop.org.
For comparison reason i run the file format identification utility
TrID ( See https://mark0.net/soft-trid-e.html). This identifies
all WBT samples as "Webmin Theme" or as variant "Webmin Theme (gif)"
with highest priority with mime type application/x-gtar by wbt.trid.xml
or wbt-gif.trid.xml. Many of the WBM samples are described as "Webmin
Module" with application/octet-stream by wbm.trid.xml. A few WBM samples
(like old-blue-theme.wbm old-blue-theme-1.0.wbm old-mscstyle3.wbm
virtual-server-mobile.wbm) are described as "Webmin Theme". The samples
are also described with lowest priority as "TAR - Tape ARchive
(directory)" with application/x-tar mime type and 1 suffix (.TAR) by
ark-tar-dir.trid.xml (See appended trid-v-webmin.txt.gz).
This tool list the used file name extension and with -v option the
related URL pointing to some file format information. The tar format is
documented, but is is used by Webmin to pack their modules or themes.
Luckily there exist a page about creating Modules and Themes on webmin
web server.
So i use these pages as reference. That informations are expressed by
comment lines inside Magdir/archive like:
# URL: https://en.wikipedia.org/wiki/Webmin
# https://webmin.com/docs/development/creating-modules/
# https://webmin.com/docs/development/creating-themes/
# Ref.: http://mark0.net/download/triddefs_xml.7z/defs/w/
# wbm.trid.xml wbt.trid.xml
So i put displaying part inside sub routine that starts like
0 name tar-webmin
>0 string x Webmin
According to documentation a module.info file is mandatory for WBM
samples. For WBT samples is written that here theme.info instead is
used. So i look for such files inside archives. The Webmin software
instead of standard suffix like GTAR or TAR use another file name suffix
like WBM or WBT. Instead of standard tar mime type i choose an user
defined one. So instead of unpacking tool an module unpacking tool for
webmin modules can be called. So the sub classification for Module or
Theme is done by next part. That looks like:
>>512 search/1767941/s /module.info Module
!:mime application/x-webmin-module
!:ext wbm
>>512 default x
>>>512 search/3149333/s /theme.info Theme
!:mime application/x-webmin-theme
!:ext wbt
>>>512 default x Module or Theme
!:mime application/x-webmin
!:ext wbm/wbt
Unfortunately a few samples (like old-blue-theme.wbm
old-blue-theme-1.0.wbm old-mscstyle3.wbm virtual-server-mobile.wbm) are
then described as WBT. After hard thinking i believe that in these
samples the wrong file name suffix WBM instead of WBT is here used. For
case that no suited info file is found then offer "third" sub class.
Most samples are packed with GNU tar. A few (1/429) samples like
cwmail.wbm are packed with POSIX tar. So show that information by next
part. That looks like:
>257 string =ustar (
>>261 ubelong =0x72202000 \bGNU tar)
>>261 ubelong 0x72003030 \bPOSIX tar)
Apparently the first archive member name[100] is directory (like:
dynbind/ ssh/ virtualmin-powerdns/ virtual-server-mobile/ vnc/ ). This
names often correlates with the module name. So show that information by
next line. That looks like:
>>0 use tar-entry
Then we see that most (419/429) *.WBM and (71/71) *.WBT samples has user
name jcameron of Webmin developer Jamie Cameron in first tar archive
member. So i can use this feature as test criterium to distinguish
webmin samples from normal tar archives.
Then in subroutine look again for characteristic WBM module info name
starting with "module.info" (like in ssh2.wbm) or characteristic WBT
theme info name starting with "theme.info" (like in:
authentic-theme-21.09.5.wbt) and for control reason show information
about that archive member. That is done by last part. That looks like:
>>512 search/1767941/s /module.info
>>>&0 search/257/s ustar
>>>>&-257 use tar-entry
>>512 search/3149333/s /theme.info\0
>>>&0 search/257/s ustar
>>>>&-257 use tar-entry
Some modules also contain language specific file (like module.info.fi).
So instead of explicitly searching for module.info which often comes
very late in archive i also accept a language variant.
After check for AVM firmware with sub routine tar-avm check for webmin
samples. Most (419/429) *.WBM (71/71) *.WBT with user name jcameron of
Webmin developer Jamie Cameron in first tar archive member are handled
by first additional test line. Afterwards for few (10/429 like
apcupsd-0.81-2.wbm csavupdate.wbm cwmail.wbm dac960.wbm etcupdate.wbm
logviewer.wbm memcached.wbm rinetd.wbm shoutcast.wbm
vacationadmin-webmin-module-1.1.2.wbm) *.WBM without user name jcameron
in 1st tar member and few (10/95 like authentic-theme-21.09.5.wbt
Mozilla-Modern.wbt virtual-server-theme-2.7.wbt
fkn-webmintheme.0.6.0.wbt) *.WBT without user name jcameron in 1st tar
member i look for filename (like module.info or theme.info) and call
suited sub routine. If afterwards nothing is matched, then i assume is
is a pure tar archive which is handled by sub routine tar-file. So this
part now becomes like:
>>>>>>>>0 ubequad&0xFFffE5eaE8ffFFff 0x2e2f6460602f0000
>>>>>>>>>0 use tar-avm
>>>>>>>>265 string jcameron
>>>>>>>>>0 use tar-webmin
>>>>>>>>0 default x
>>>>>>>>>512 search/210965/s e.info\0
>>>>>>>>>>0 use tar-webmin
>>>>>>>>>0 default x
>>>>>>>>>>0 use tar-file
After applying the above mentioned modifications by patch
file-archive-webmin.diff then my webmin samples are more precisely
described. This now with -e tar option looks like:
FRITZ.Box_7520_B-07.57.image: AVM FRITZ!Box firmware HW276
(FRITZ!Box 7520 B),
version 07.57, directory
./var/, mode 0040755, uid
0000000, gid 0000000, size 00000000000,
seconds 14474476653, file
./var/content, mode 0100644, uid
0000000, gid 0000000, size 00000000543,
seconds 14474476653
apcupsd-0.81-2.wbm: Webmin Module (GNU tar), directory
apcupsd/, mode 0040700, uid
0000000, gid 0000001, size 00000000000,
seconds 07172756000, user
root, group bin, file
apcupsd/module.info, mode 0100600, uid
0000000, gid 0000001, size 00000000233,
seconds 07165100524, user
root, group bin
authentic-theme-21.09.5.wbt: Webmin Theme (GNU tar), directory
authentic-theme/, mode 0000755, uid
0000765, gid 0000024, size 00000000000,
seconds 14522234321, user
user, group staff, file
authentic-theme/theme.info,
mode 0000644, uid
0000765, gid 0000024, size 00000000651,
seconds 14522531211, user
user, group staff
cwmail.wbm: Webmin Module (POSIX tar), directory
cwmail, mode 000755 , uid
000016 , gid 000004 , size 00000000000 ,
seconds 07102263376, user
grund, group bin, devmin 476230 , file
cwmail/module.info, mode 000755 , uid
000016 , gid 000004 , size 00000000603 ,
seconds 07101263313, user
grund, group bin, devmin 475434
etcupdate.wbm: Webmin Module (GNU tar), directory
etcupdate/, mode 0000755, uid
0000000, gid 0000000, size 00000000000,
seconds 10300072753, user
root, group root, file
etcupdate/module.info, mode 0000644, uid
0000000, gid 0000000, size 00000000142,
seconds 10300351140, user
root, group root
link.wbm: Webmin Module (GNU tar), directory
link/, mode 0040775, uid
0005671, gid 0000012, size 00000000000,
seconds 07627514126, user
jcameron, group wheel, file
link/module.info, mode 0100664, uid
0005671, gid 0000012, size 00000000045,
seconds 07627513435, user
jcameron, group wheel
lpadmin.wbm: Webmin Module (GNU tar), directory
lpadmin/, mode 0000755, uid
0005671, gid 0000012, size 00000000000,
seconds 14513515572, user
jcameron, group wheel, file
lpadmin/module.info.fi, mode 0000664, uid
0005671, gid 0000012, size 00000000000,
seconds 14513515572, user
jcameron, group wheel
memcached.wbm: Webmin Module (GNU tar), directory
memcached/, mode 0000755, uid
0001750, gid 0000144, size 00000000000,
seconds 11671622453, user
user, group users, file
memcached/module.info, mode 0000755, uid
0001750, gid 0000144, size 00000000103,
seconds 11671621611, user
user, group users
old-blue-theme-1.0.wbm: Webmin Theme (GNU tar), directory
old-blue-theme/, mode 0000755, uid
0005671, gid 0000012, size 00000000000,
seconds 12457303450, user
jcameron, group wheel, file
old-blue-theme/theme.info,
mode 0000644, uid
0005671, gid 0000012, size 00000000121,
seconds 12457303450, user
jcameron, group wheel
old-blue-theme.wbm: Webmin Theme (GNU tar), directory
old-blue-theme/, mode 0040755, uid
0005671, gid 0000012, size 00000000000,
seconds 10533114330, user
jcameron, group wheel, file
old-blue-theme/theme.info,
mode 0100664, uid
0005671, gid 0000012, size 00000000172,
seconds 10533114330, user
jcameron, group wheel
old-mscstyle3.wbm: Webmin Theme (GNU tar), directory
old-mscstyle3/, mode 0040755, uid
0005671, gid 0000000, size 00000000000,
seconds 10343426200, user
jcameron, group root, file
old-mscstyle3/theme.info,
mode 0100644, uid
0005671, gid 0000000, size 00000000121,
seconds 10343426200, user
jcameron, group root
virtual-server-mobile.wbm: Webmin Theme (GNU tar), directory
virtual-server-mobile/, mode 0040775, uid
0005671, gid 0000012, size 00000000000,
seconds 11047144366, user
jcameron, group wheel, file
virtual-server-mobile/theme.info,
mode 0100664, uid
0005671, gid 0000012, size 00000000136,
seconds 11047144365, user
jcameron, group wheel
virtual-server-theme-3.4.wbt: Webmin Theme (GNU tar), directory
virtual-server-theme/, mode 0040775, uid
0005671, gid 0000012, size 00000000000,
seconds 10562743567, user
jcameron, group wheel, file
virtual-server-theme/theme.info,
mode 0100664, uid
0005671, gid 0000012, size 00000000074,
seconds 10562743567, user
jcameron, group wheel
webmin.wbm: Webmin Module (GNU tar), directory
webmin/, mode 0000755, uid
0005671, gid 0000012, size 00000000000,
seconds 14513515571, user
jcameron, group wheel, file
webmin/module.info.fi, mode 0000664, uid
0005671, gid 0000012, size 00000000000,
seconds 14513515571, user
jcameron, group wheel
I hope my diff file can be applied in future version of file
utility.
With best wishes,
Jörg Jenderek
--
Jörg Jenderek
-------------- next part --------------
--
File mailing list
File at astron.com
https://mailman.astron.com/mailman/listinfo/file
-------------- next part --------------
A non-text attachment was scrubbed...
Name: trid-v-image.txt.gz
Type: application/x-gzip
Size: 1015 bytes
Desc: not available
URL: <https://mailman.astron.com/pipermail/file/attachments/20240722/d66a2642/attachment.bin>
-------------- next part --------------
--- file-master/magic/Magdir/archive.old 2024-07-19 14:31:25.882146200 +0200
+++ file-master/magic/Magdir/archive 2024-07-22 18:03:40.672566100 +0200
@@ -53,6 +53,17 @@
# >>>>>>>>>>0 use tar-avm
+# most (419/429) *.WBM (71/71) *.WBT with user name jcameron of Webmin developer Jamie Cameron in first tar archive member
+>>>>>>>>265 string jcameron
+>>>>>>>>>0 use tar-webmin
# if 1st member name without digits and without used image suffix, without *.ovf,
-# ./var/ , ./lte/ and TpmEmuTpms/ then it is a pure TAR archive
+# ./var/ , ./lte/ and TpmEmuTpms/ then it is a pure TAR archive or Webmin without jcameron user name
>>>>>>>>0 default x
->>>>>>>>>0 use tar-file
+# few (10/429) *.WBM without user name jcameron in 1st tar member but with WBM module.info name like:
+# apcupsd-0.81-2.wbm csavupdate.wbm cwmail.wbm dac960.wbm etcupdate.wbm logviewer.wbm memcached.wbm rinetd.wbm shoutcast.wbm vacationadmin-webmin-module-1.1.2.wbm
+# few (10/95) *.WBT without user name jcameron in 1st tar member but with WBT theme.info name like:
+# authentic-theme-21.09.5.wbt Mozilla-Modern.wbt virtual-server-theme-2.7.wbt fkn-webmintheme.0.6.0.wbt
+>>>>>>>>>512 search/210965/s e.info\0
+>>>>>>>>>>0 use tar-webmin
+# pure TAR
+>>>>>>>>>0 default x
+>>>>>>>>>>0 use tar-file
# Note: called "TAR - Tape ARchive" by TrID, "Tape Archive Format" by DROID via PUID x-fmt/265
@@ -259,2 +270,52 @@
>>512 use tar-entry
+# Summary: Webmin Module or Theme
+# From: Joerg Jenderek
+# URL: https://en.wikipedia.org/wiki/Webmin
+# https://webmin.com/docs/development/creating-modules/
+# https://webmin.com/docs/development/creating-themes/
+# Reference: http://mark0.net/download/triddefs_xml.7z/defs/w/wbm.trid.xml
+# http://mark0.net/download/triddefs_xml.7z/defs/w/wbt.trid.xml
+# http://mark0.net/download/triddefs_xml.7z/defs/w/wbt-gif.trid.xml
+# Note: called "Webmin Module" "Webmin Theme" by TrID
+# most verfied by 7-Zip `7z l -ttar *.wbm | grep "\module.info"` and
+# `7z l -ttar *.wbt | grep "\theme.info"`
+0 name tar-webmin
+>0 string x Webmin
+# Webmin module or theme
+>>512 search/1767941/s /module.info Module
+!:mime application/x-webmin-module
+!:ext wbm
+# According to documentation module.info is mandatory but instead theme.info is found in
+# old-blue-theme.wbm old-blue-theme-1.0.wbm old-mscstyle3.wbm virtual-server-mobile.wbm
+# GRR: maybe here wrong file name suffix WBM instead of WBT
+>>512 default x
+>>>512 search/3149333/s /theme.info Theme
+!:mime application/x-webmin-theme
+!:ext wbt
+# next 3 lines should not happen
+>>>512 default x Module or Theme
+!:mime application/x-webmin
+!:ext wbm/wbt
+# GNU or POSIX tar
+>257 string =ustar (
+# 2 space characters followed by a null for GNU variant for most (428/429) WBM samples
+>>261 ubelong =0x72202000 \bGNU tar)
+#!:mime application/x-gtar
+# UStar version variant with ASCII "00" as in few (1/429) samples like cwmail.wbm
+>>261 ubelong 0x72003030 \bPOSIX tar)
+#!:mime application/x-ustar
+#>>>156 ubyte x tar archive
+# Apparently first archive member name[100] is directory like: dynbind/ ssh/ virtualmin-powerdns/ virtual-server-mobile/ vnc/
+>>0 use tar-entry
+# look for characteristic WBM module info name starting with "module.info" for language variant like in: ssh2.wbm
+>>512 search/1767941/s /module.info
+# look for TAR magic of WBM archive module info
+>>>&0 search/257/s ustar
+# show details for WBM archive member module info
+>>>>&-257 use tar-entry
+# look for characteristic WBT theme info name with "theme.info" like in: authentic-theme-21.09.5.wbt
+>>512 search/3149333/s /theme.info\0
+# look for TAR magic of WBT archive theme info
+>>>&0 search/257/s ustar
+>>>>&-257 use tar-entry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: file-archive-webmin.diff.sig
Type: application/octet-stream
Size: 1722 bytes
Desc: not available
URL: <https://mailman.astron.com/pipermail/file/attachments/20240722/d66a2642/attachment.obj>
More information about the File
mailing list