[File] [PATCH] Magdir/images, mcrypt for NetCDF Data Format, mcrypt encrypted *.NC
Jörg Jenderek
joerg.jen.der.ek at gmx.net
Tue Feb 1 23:20:54 UTC 2022
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Hello,
some days ago i handled some NetCDF Data samples with file name
extension NC.
When running file command version 5.41 on such examples and related
files i get an output like:
19981111_0045.nc: NetCDF Data Format data
HRDL_iop12-example.nc: NetCDF Data Format data
README.txt.nc: mcrypt 2.5 encrypted data,
algorithm: rijndael-128,
keysize: 32 bytes, mode: cbc,
bad_cdf5_begin.nc: data
dmake.nc: ASCII text, with CR line terminators
fmt-282-signature-id-298.nc: NetCDF Data Format data
fmt-283-signature-id-299.nc: NetCDF Data Format data (64-bit offset)
rhum.2003.nc: NetCDF Data Format data
test_echam_spectral.nc: NetCDF Data Format data
tst_small_64bit.nc: NetCDF Data Format data (64-bit offset)
Furthermore with -i option only generic application/octet-stream or
text/plain is shown. With option --extension only 3 byte sequence ???
is shown.
For comparison reason i run the file format identification utility
TrID ( See https://mark0.net/soft-trid-e.html). Most examples are
described as "NetCDF Network Common Data Form" with mime type
application/netcdf by netcdf.trid.xml. The example README.txt.nc is
described as "mcrypt encrypted (v2.5)" with application/octet-stream
by nc-mcrypt.trid.xml (See appended nc-trid-v.txt.gz).
Luckily TrID tool displays correct file name extension NC for
inspected samples. This list with -v option the related URL pointing
to used file information.
For comparison reason i also run the file format identification
utility DROID ( See https://sourceforge.net/projects/droid/). This
identifies most as "netCDF-3 Classic" by PUID fmt/282. Example
tst_small_64bit.nc is described as "netCDF-3 64-bit" by PUID fmt/283.
As mime type here application/netcdf and application/x-netcdf is
shown. (See appended DROID-nc.csv.gz)
That informations are expressed by comment lines inside Magdir/images
like:
# URL: https://en.wikipedia.org/wiki/NetCDF
# http://fileformats.archiveteam.org/wiki/NetCDF
# Ref.: http://mark0.net/download/triddefs_xml.7z
# defs/n/netcdf.trid.xml
# https://www.loc.gov/preservation/digital/formats/
# fdd/fdd000330.shtml
The description happen by lines like:
0 string CDF\001 NetCDF Data Format data
0 string CDF\002 NetCDF Data Format data (64-bit offset)
This now becomes like:
0 string CDF\001
>3 leshort >0 NetCDF Data Format data
!:mime application/x-netcdf
!:ext nc
0 string CDF\002
>3 leshort >0 NetCDF Data Format data (64-bit offset)
!:mime application/x-netcdf
!:ext nc
The DROID signatures contain just the 4 magic bytes. So the
2 examples fmt-282-signature-id-298.nc and
fmt-283-signature-id-299.nc are identified as NetCDF, but in
reality these examples contain no data. To skip these examples i
test also for bytes afterwards by second line. Some sites us the
mime type application/netcdf, but i found no such official
registered type.
So i apply the user defined one. According to www.file-extensions.org
cdf was used as file name extension, but in 1994 changed to NC to
avoid a clash with other file formats using also THE CDF file name
extension. I my self found no example with CDF extension. So i
display only the NC extension.
That informations are expressed by comment lines inside Magdir/mcrypt
like:
# URL: https://en.wikipedia.org/wiki/Mcrypt
# http://fileformats.archiveteam.org/wiki/MCrypt
# Reference: http://mark0.net/download/triddefs_xml.7z
# defs/n/nc-mcrypt.trid.xml
The description happen by lines like:
0 string \0m\3 mcrypt 2.5 encrypted data,
This now becomes like:
0 string \0m\3 mcrypt 2.5 encrypted data,
!:mime application/x-crypt-nc
!:ext nc
Instead of generic mime type application/octet-stream i display a
user defined one.
After applying the above mentioned modifications by patches
file-5.41-mcrypt-nc.diff and file-5.41-images-nc.diff
then all my inspected examples are now described as before, but
some misidentification vanish. This now looks like:
19981111_0045.nc: NetCDF Data Format data
HRDL_iop12-example.nc: NetCDF Data Format data
README.txt.nc: mcrypt 2.5 encrypted data,
algorithm: rijndael-128,
keysize: 32 bytes, mode: cbc,
bad_cdf5_begin.nc: data
dmake.nc: ASCII text, with CR line terminators
fmt-282-signature-id-298.nc: data
fmt-283-signature-id-299.nc: data
rhum.2003.nc: NetCDF Data Format data
test_echam_spectral.nc: NetCDF Data Format data
tst_small_64bit.nc: NetCDF Data Format data (64-bit offset)
I hope my diff files can be applied in future version of file
utility. Newer versions are based on the Hierarchical Data Format
version 5. So these must be handled in a future session.
With best wishes
Jörg Jenderek
- --
Jörg Jenderek
-----BEGIN PGP SIGNATURE-----
Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/
iF0EARECAB0WIQS5/qNWKD4ASGOJGL+v8rHJQhrU1gUCYfnAVgAKCRCv8rHJQhrU
1l73AJoDWuY2YzoVyZ/L+irKEJXqq37XHQCgg1TLp65Sx21fcr3tBMifo7Ak9mQ=
=LR55
-----END PGP SIGNATURE-----
-------------- next part --------------
--
File mailing list
File at astron.com
https://mailman.astron.com/mailman/listinfo/file
-------------- next part --------------
--- file-5.41/magic/Magdir/mcrypt.old 2019-02-22 13:06:34.000000000 +0100
+++ file-5.41/magic/Magdir/mcrypt 2022-02-01 20:27:51.522717300 +0100
@@ -5,3 +5,11 @@
# mcrypt: file(1) magic for mcrypt 2.2.x;
+# URL: https://en.wikipedia.org/wiki/Mcrypt
+# http://fileformats.archiveteam.org/wiki/MCrypt
+# Reference: http://mark0.net/download/triddefs_xml.7z/defs/n/nc-mcrypt.trid.xml
+# Update: Joerg Jenderek
+# Note: called by TrID "mcrypt encrypted (v2.5)"
0 string \0m\3 mcrypt 2.5 encrypted data,
+#!:mime application/octet-stream
+!:mime application/x-crypt-nc
+!:ext nc
>4 string >\0 algorithm: %s,
@@ -10,3 +18,9 @@
+# Reference: http://mark0.net/download/triddefs_xml.7z/defs/n/nc-mcrypt-22.trid.xml
+# Note: called by TrID "mcrypt encrypted (v2.2)"
0 string \0m\2 mcrypt 2.2 encrypted data,
+#!:mime application/octet-stream
+!:mime application/x-crypt-nc
+# no example
+!:ext nc
>3 byte 0 algorithm: blowfish-448,
-------------- next part --------------
A non-text attachment was scrubbed...
Name: file-5.41-mcrypt-nc.diff.sig
Type: application/octet-stream
Size: 614 bytes
Desc: not available
URL: <https://mailman.astron.com/pipermail/file/attachments/20220202/177e86b0/attachment.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: nc-trid-v.txt.gz
Type: application/x-gzip
Size: 535 bytes
Desc: not available
URL: <https://mailman.astron.com/pipermail/file/attachments/20220202/177e86b0/attachment.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: nc-DROID.csv.gz
Type: application/x-gzip
Size: 629 bytes
Desc: not available
URL: <https://mailman.astron.com/pipermail/file/attachments/20220202/177e86b0/attachment-0001.bin>
-------------- next part --------------
--- file-5.41/magic/Magdir/images.old 2021-10-18 16:20:03.000000000 +0200
+++ file-5.41/magic/Magdir/images 2022-02-02 00:17:27.552345400 +0100
@@ -1660,8 +1660,31 @@
# From: Tom Hilinski <tom.hilinski at comcast.net>
-# https://www.unidata.ucar.edu/software/netcdf/
-0 string CDF\001 NetCDF Data Format data
+# Update: Joerg Jenderek
+# URL: https://en.wikipedia.org/wiki/NetCDF
+# http://fileformats.archiveteam.org/wiki/NetCDF
+# Reference: http://mark0.net/download/triddefs_xml.7z/defs/n/netcdf.trid.xml
+# https://www.loc.gov/preservation/digital/formats/fdd/fdd000330.shtml
+# Note: called "NetCDF Network Common Data Form" by TrID and
+# "netCDF-3 Classic" by DROID via PUID fmt/282
+# https://www.unidata.ucar.edu/packages/netcdf/
+0 string CDF\001
+# skip DROID fmt-282-signature-id-298.nc by test for more content bytes
+>3 uleshort >0 NetCDF Data Format data
+#!:mime application/netcdf
+# https://reposcope.com/mimetype/application/x-netcdf
+!:mime application/x-netcdf
+!:ext nc
+# https://fileinfo.com/extension/cdf
+# https://www.file-extensions.org/cdf-file-extension-unidata-network-common-data-form
+# in 1994 changed from CDF to NC file extension avoid a clash with other file formats
+#!:ext nc/cdf
# 64-bit offset netcdf Classic https://www.unidata.ucar.edu/software/netcdf/docs/file_format_specifications
-0 string CDF\002 NetCDF Data Format data (64-bit offset)
+# Note: called "netCDF-3 64-bit" by DROID via PUID fmt/283
+0 string CDF\002
+# skip DROID fmt-283-signature-id-299.nc by test for more content bytes
+>3 uleshort >0 NetCDF Data Format data (64-bit offset)
+#!:mime application/netcdf
+!:mime application/x-netcdf
+!:ext nc
# From: Michael Liu
-------------- next part --------------
A non-text attachment was scrubbed...
Name: file-5.41-images-nc.diff.sig
Type: application/octet-stream
Size: 939 bytes
Desc: not available
URL: <https://mailman.astron.com/pipermail/file/attachments/20220202/177e86b0/attachment-0001.obj>
More information about the File
mailing list