[File] Building a specific version of magic.mgc ?

Christos Zoulas christos at zoulas.com
Wed Jun 2 13:40:29 UTC 2021 

Great, that is indeed the best way to go since it avoids mixing versions and having a Frankenstein system.

christos

> On Jun 2, 2021, at 9:33 AM, Frédéric Nass <frederic.nass at univ-lorraine.fr> wrote:
> 
> Christos,
> 
> Since Amavis now uses the /usr/share/misc/magic.mgc file with the Perl module "File::LibMagic" (instead of calling the system File command) I had no choice but to use a new /usr/share/misc/magic.mgc file and as a consequence the latest File binary.
> So I did 'make install' and pointed each one of these to files to their /usr/local/... corresponding versions.
> 
> Attachments are now recognized and delivered to their recipients correctly.
> 
> Thanks for your precious help,
> 
> Best regards,
> 
> Frédéric.
> 
> Le 02/06/2021 à 10:42, Frédéric Nass a écrit :
>> In addition, it looks like the magic.mgc file was built from the existing /usr/share/misc/magic.mgc file and not the /etc/magic file that I generated with the 'cat' command, as when I move this /usr/share/misc/magic.mgc file to another place, the build fails.
>> 
>> [...]
>> /etc/magic, 33859: Warning: indirect offset type `q' invalid
>> /etc/magic, 33863: Warning: indirect offset type `q' invalid
>> /etc/magic, 33871: Warning: indirect offset type `q' invalid
>> file: could not find any valid magic files!
>> 
>> Regards,
>> 
>> Frédéric.
>> 
>> Le 02/06/2021 à 10:10, Frédéric Nass a écrit :
>>> Hi Christos,
>>> 
>>> Thank you for your quick answer. I really appreciate it. Here's what I did:
>>> 
>>> $ cd /tmp/
>>> $ git clone https://github.com/file/file
>>> $ cat /tmp/file/magic/Magdir/* >> /etc/magic
>>> $ file --compile
>>> 
>>> I then got a lot of warnings:
>>> 
>>> $ file --compile
>>> /etc/magic, 4182: Warning: indirect offset type `Q' invalid
>>> /etc/magic, 4191: Warning: indirect offset type `Q' invalid
>>> /etc/magic, 4200: Warning: indirect offset type `Q' invalid
>>> /etc/magic, 4208: Warning: indirect offset type `Q' invalid
>>> /etc/magic, 4418: Warning: type `guid 33000890-E5B1-11CF-89F4-00A0C90349CB' invalid
>>> /etc/magic, 4419: Warning: type `guid D6E229D3-35DA-11D1-9034-00A0C90349BE ASF_Index_Object' invalid
>>> /etc/magic, 4420: Warning: type `guid FEB103F8-12AD-4C64-840F-2A1D2F7AD48C ASF_Media_Object_Index_Object' invalid
>>> /etc/magic, 4421: Warning: type `guid 3CB73FD0-0C4A-4803-953D-EDF7B6228F0C ASF_Timecode_Index_Object' invalid
>>> /etc/magic, 4424: Warning: type `guid 8CABDCA1-A947-11CF-8EE4-00C00C205365 ' invalid
>>> /etc/magic, 4427: Warning: type `guid B7DC0791-A9B7-11CF-8EE6-00C00C205365' invalid
>>> /etc/magic, 4434: Warning: type `guid F8699E40-5B4D-11CF-A8FD-00805F5C442B \b, Audio Media (' invalid
>>> /etc/magic, 4442: Warning: type `guid BC19EFC0-5B4D-11CF-A8FD-00805F5C442B \b, Video Media (' invalid
>>> /etc/magic, 4460: Warning: type `guid 5FBF03B5-A92E-11CF-8EE3-00C00C205365' invalid
>>> /etc/magic, 4462: Warning: type `guid 86D15240-311D-11D0-A3A4-00A0C90348F6' invalid
>>> /etc/magic, 4463: Warning: type `guid 1EFB1A30-0B62-11D0-A39B-00A0C90348F6 ASF_Script_Command_Object' invalid
>>> /etc/magic, 4464: Warning: type `guid F487CD01-A951-11CF-8EE6-00C00C205365 ASF_Marker_Object' invalid
>>> /etc/magic, 4465: Warning: type `guid D6E229DC-35DA-11D1-9034-00A0C90349BE ASF_Bitrate_Mutual_Exclusion_Object' invalid
>>> /etc/magic, 4466: Warning: type `guid 75B22635-668E-11CF-A6D9-00AA0062CE6C ASF_Error_Correction_Object' invalid
>>> /etc/magic, 4468: Warning: type `guid 75B22633-668E-11CF-A6D9-00AA0062CE6C' invalid
>>> /etc/magic, 4474: Warning: type `guid D2D0A440-E307-11D2-97F0-00A0C95EA850 ASF_Extended_Content_Description_Object' invalid
>>> /etc/magic, 4475: Warning: type `guid 2211B3FA-BD23-11D2-B4B7-00A0C955FC6E ASF_Content_Branding_Object' invalid
>>> /etc/magic, 4476: Warning: type `guid 7BF875CE-468D-11D1-8D82-006097C9A2B2 ASF_Stream_Bitrate_Properties_Object' invalid
>>> /etc/magic, 4477: Warning: type `guid 2211B3FB-BD23-11D2-B4B7-00A0C955FC6E ASF_Content_Encryption_Object' invalid
>>> /etc/magic, 4478: Warning: type `guid 298AE614-2622-4C17-B935-DAE07EE9289C ASF_Extended_Content_Encryption_Object' invalid
>>> /etc/magic, 4479: Warning: type `guid 2211B3FC-BD23-11D2-B4B7-00A0C955FC6E ASF_Digital_Signature_Object' invalid
>>> /etc/magic, 4481: Warning: type `guid 1806D474-CADF-4509-A4BA-9AABCB96AAE8' invalid
>>> /etc/magic, 4482: Warning: type `guid 14E6A5CB-C672-4332-8399-A96952065B5A ASF_Extended_Stream_Properties_Object' invalid
>>> /etc/magic, 4483: Warning: type `guid A08649CF-4775-4670-8A16-6E35357566CD ASF_Advanced_Mutual_Exclusion_Object' invalid
>>> /etc/magic, 4484: Warning: type `guid D1465A40-5A79-4338-B71B-E36B8FD6C249 ASF_Group_Mutual_Exclusion_Object' invalid
>>> /etc/magic, 4485: Warning: type `guid D4FED15B-88D3-454F-81F0-ED5C45999E24 ASF_Stream_Prioritization_Object' invalid
>>> /etc/magic, 4486: Warning: type `guid A69609E6-517B-11D2-B6AF-00C04FD908E9 ASF_Bandwidth_Sharing_Object' invalid
>>> /etc/magic, 4487: Warning: type `guid 7C4346A9-EFE0-4BFC-B229-393EDE415C85 ASF_Language_List_Object' invalid
>>> /etc/magic, 4488: Warning: type `guid C5F8CBEA-5BAF-4877-8467-AA8C44FA4CCA ASF_Metadata_Object' invalid
>>> /etc/magic, 4489: Warning: type `guid 44231C94-9498-49D1-A141-1D134E457054 ASF_Metadata_Library_Object' invalid
>>> /etc/magic, 4490: Warning: type `guid D6E229DF-35DA-11D1-9034-00A0C90349BE ASF_Index_Parameters_Object' invalid
>>> /etc/magic, 4491: Warning: type `guid 6B203BAD-3F11-48E4-ACA8-D7613DE2CFA7 ASF_Media_Object_Index_Parameters_Object' invalid
>>> /etc/magic, 4492: Warning: type `guid F55E496D-9797-4B5D-8C8B-604DFE9BFB24 ASF_Timecode_Index_Parameters_Object' invalid
>>> /etc/magic, 4493: Warning: type `guid 26F18B5D-4584-47EC-9F5F-0E651F0452C9 ASF_Compatibility_Object' invalid
>>> /etc/magic, 4494: Warning: type `guid 43058533-6981-49E6-9B74-AD12CB86D58C ASF_Advanced_Content_Encryption_Object' invalid
>>> /etc/magic, 4495: Warning: type `guid 59DACFC0-59E6-11D0-A3AC-00A0C90348F6 ASF_Command_Media' invalid
>>> /etc/magic, 4496: Warning: type `guid B61BE100-5B4E-11CF-A8FD-00805F5C44 ASF_JFIF_Media' invalid
>>> /etc/magic, 4497: Warning: type `guid 35907DE0-E415-11CF-A917-00805F5C442B ASF_Degradable_JPEG_Media' invalid
>>> /etc/magic, 4498: Warning: type `guid 91BD222C-F21C-497A-8B6D-5AA86BFC0185 ASF_File_Transfer_Media' invalid
>>> /etc/magic, 4499: Warning: type `guid 3AFB65E2-47EF-40F2-AC2C-70A90D71D343 ASF_Binary_Media' invalid
>>> /etc/magic, 4500: Warning: type `guid 776257D4-C627-41CB-8F81-7AC7FF1C40CC ASF_Web_Stream_Media_Subtype' invalid
>>> /etc/magic, 4501: Warning: type `guid DA1E6B13-8359-4050-B398-388E965BF00C ASF_Web_Stream_Format' invalid
>>> /etc/magic, 4502: Warning: type `guid 20FB5700-5B55-11CF-A8FD-00805F5C442B ASF_No_Error_Correction' invalid
>>> /etc/magic, 4503: Warning: type `guid BFC3CD50-618F-11CF-8BB2-00AA00B4E220 ASF_Audio_Spread' invalid
>>> /etc/magic, 4504: Warning: type `guid ABD3D211-A9BA-11cf-8EE6-00C00C205365 ASF_Reserved_1' invalid
>>> /etc/magic, 4505: Warning: type `guid 7A079BB6-DAA4-4e12-A5CA-91D38DC11A8D ASF_Content_Encryption_System_Windows_Media_DRM' invalid
>>> /etc/magic, 4507: Warning: type `guid 86D15241-311D-11D0-A3A4-00A0C90348F6 ASF_Reserved_2' invalid
>>> /etc/magic, 4508: Warning: type `guid 4B1ACBE3-100B-11D0-A39B-00A0C90348F6 ASF_Reserved_3' invalid
>>> /etc/magic, 4509: Warning: type `guid 4CFEDB20-75F6-11CF-9C0F-00A0C90349CB ASF_Reserved_4' invalid
>>> /etc/magic, 4510: Warning: type `guid D6E22A00-35DA-11D1-9034-00A0C90349BE ASF_Mutex_Language' invalid
>>> /etc/magic, 4511: Warning: type `guid D6E22A01-35DA-11D1-9034-00A0C90349BE ASF_Mutex_Bitrate' invalid
>>> /etc/magic, 4512: Warning: type `guid D6E22A02-35DA-11D1-9034-00A0C90349BE ASF_Mutex_Unknown' invalid
>>> /etc/magic, 4513: Warning: type `guid AF6060AA-5197-11D2-B6AF-00C04FD908E9 ASF_Bandwidth_Sharing_Exclusive' invalid
>>> /etc/magic, 4514: Warning: type `guid AF6060AB-5197-11D2-B6AF-00C04FD908E9 ASF_Bandwidth_Sharing_Partial' invalid
>>> /etc/magic, 4515: Warning: type `guid 399595EC-8667-4E2D-8FDB-98814CE76C1E ASF_Payload_Extension_System_Timecode' invalid
>>> /etc/magic, 4516: Warning: type `guid E165EC0E-19ED-45D7-B4A7-25CBD1E28E9B ASF_Payload_Extension_System_File_Name' invalid
>>> /etc/magic, 4517: Warning: type `guid D590DC20-07BC-436C-9CF7-F3BBFBF1A4DC ASF_Payload_Extension_System_Content_Type' invalid
>>> /etc/magic, 4518: Warning: type `guid 1B1EE554-F9EA-4BC8-821A-376B74E4C4B8 ASF_Payload_Extension_System_Pixel_Aspect_Ratio' invalid
>>> /etc/magic, 4519: Warning: type `guid C6BD9450-867F-4907-83A3-C77921B733AD ASF_Payload_Extension_System_Sample_Duration' invalid
>>> /etc/magic, 4520: Warning: type `guid 6698B84E-0AFA-4330-AEB2-1C0A98D7A44D ASF_Payload_Extension_System_Encryption_Sample_ID' invalid
>>> /etc/magic, 4521: Warning: type `guid 00E1AF06-7BEC-11D1-A582-00C04FC29CFB ASF_Payload_Extension_System_Degradable_JPEG' invalid
>>> /etc/magic, 4527: Warning: indirect offset type `q' invalid
>>> /etc/magic, 4531: Warning: type `guid 75B22630-668E-11CF-A6D9-00AA0062CE6C    Microsoft ASF' invalid
>>> /etc/magic, 4532: Warning: Current entry does not yet have a description for adding a MIME type
>>> /etc/magic, 4537: Warning: indirect offset type `q' invalid
>>> /etc/magic, 7420: Warning: string modifier `f' invalid
>>> /etc/magic, 7421: Warning: Current entry does not yet have a description for adding a MIME type
>>> /etc/magic, 7422: Warning: string modifier `f' invalid
>>> /etc/magic, 7423: Warning: Current entry does not yet have a description for adding a MIME type
>>> /etc/magic, 7425: Warning: string modifier `f' invalid
>>> /etc/magic, 7426: Warning: Current entry does not yet have a description for adding a MIME type
>>> /etc/magic, 7429: Warning: string modifier `f' invalid
>>> /etc/magic, 7430: Warning: Current entry does not yet have a description for adding a MIME type
>>> /etc/magic, 7431: Warning: string modifier `f' invalid
>>> /etc/magic, 7432: Warning: Current entry does not yet have a description for adding a MIME type
>>> /etc/magic, 7434: Warning: string modifier `f' invalid
>>> /etc/magic, 7435: Warning: Current entry does not yet have a description for adding a MIME type
>>> /etc/magic, 7436: Warning: string modifier `f' invalid
>>> /etc/magic, 7437: Warning: Current entry does not yet have a description for adding a MIME type
>>> /etc/magic, 7438: Warning: string modifier `f' invalid
>>> /etc/magic, 7439: Warning: Current entry does not yet have a description for adding a MIME type
>>> /etc/magic, 7440: Warning: string modifier `f' invalid
>>> /etc/magic, 7441: Warning: Current entry does not yet have a description for adding a MIME type
>>> /etc/magic, 7445: Warning: string modifier `f' invalid
>>> /etc/magic, 7446: Warning: Current entry does not yet have a description for adding a MIME type
>>> /etc/magic, 7447: Warning: string modifier `f' invalid
>>> /etc/magic, 7448: Warning: Current entry does not yet have a description for adding a MIME type
>>> /etc/magic, 7449: Warning: string modifier `f' invalid
>>> /etc/magic, 7450: Warning: Current entry does not yet have a description for adding a MIME type
>>> /etc/magic, 7454: Warning: string modifier `f' invalid
>>> /etc/magic, 7455: Warning: Current entry does not yet have a description for adding a MIME type
>>> /etc/magic, 7456: Warning: string modifier `f' invalid
>>> /etc/magic, 7457: Warning: Current entry does not yet have a description for adding a MIME type
>>> /etc/magic, 7458: Warning: string modifier `f' invalid
>>> /etc/magic, 7459: Warning: Current entry does not yet have a description for adding a MIME type
>>> /etc/magic, 7460: Warning: string modifier `f' invalid
>>> /etc/magic, 7461: Warning: Current entry does not yet have a description for adding a MIME type
>>> /etc/magic, 7462: Warning: string modifier `f' invalid
>>> /etc/magic, 7463: Warning: Current entry does not yet have a description for adding a MIME type
>>> /etc/magic, 7464: Warning: string modifier `f' invalid
>>> /etc/magic, 7465: Warning: Current entry does not yet have a description for adding a MIME type
>>> /etc/magic, 7466: Warning: string modifier `f' invalid
>>> /etc/magic, 7467: Warning: Current entry does not yet have a description for adding a MIME type
>>> /etc/magic, 7468: Warning: string modifier `f' invalid
>>> /etc/magic, 7469: Warning: Current entry does not yet have a description for adding a MIME type
>>> /etc/magic, 7472: Warning: string modifier `f' invalid
>>> /etc/magic, 7473: Warning: Current entry does not yet have a description for adding a MIME type
>>> /etc/magic, 7475: Warning: string modifier `f' invalid
>>> /etc/magic, 7476: Warning: Current entry does not yet have a description for adding a MIME type
>>> /etc/magic, 7477: Warning: string modifier `f' invalid
>>> /etc/magic, 7478: Warning: Current entry does not yet have a description for adding a MIME type
>>> /etc/magic, 7482: Warning: string modifier `f' invalid
>>> /etc/magic, 7485: Warning: string modifier `f' invalid
>>> /etc/magic, 7486: Warning: Current entry does not yet have a description for adding a MIME type
>>> /etc/magic, 7487: Warning: string modifier `f' invalid
>>> /etc/magic, 7488: Warning: Current entry does not yet have a description for adding a MIME type
>>> /etc/magic, 7489: Warning: string modifier `f' invalid
>>> /etc/magic, 7490: Warning: Current entry does not yet have a description for adding a MIME type
>>> /etc/magic, 7491: Warning: string modifier `f' invalid
>>> /etc/magic, 7492: Warning: Current entry does not yet have a description for adding a MIME type
>>> /etc/magic, 7493: Warning: string modifier `f' invalid
>>> /etc/magic, 7494: Warning: Current entry does not yet have a description for adding a MIME type
>>> /etc/magic, 7495: Warning: string modifier `f' invalid
>>> /etc/magic, 7496: Warning: Current entry does not yet have a description for adding a MIME type
>>> /etc/magic, 7497: Warning: string modifier `f' invalid
>>> /etc/magic, 7498: Warning: Current entry does not yet have a description for adding a MIME type
>>> /etc/magic, 7499: Warning: string modifier `f' invalid
>>> /etc/magic, 7500: Warning: Current entry does not yet have a description for adding a MIME type
>>> /etc/magic, 7501: Warning: string modifier `f' invalid
>>> /etc/magic, 7502: Warning: Current entry does not yet have a description for adding a MIME type
>>> /etc/magic, 7506: Warning: string modifier `f' invalid
>>> /etc/magic, 7507: Warning: Current entry does not yet have a description for adding a MIME type
>>> /etc/magic, 7508: Warning: string modifier `f' invalid
>>> /etc/magic, 7509: Warning: Current entry does not yet have a description for adding a MIME type
>>> /etc/magic, 7510: Warning: string modifier `f' invalid
>>> /etc/magic, 7511: Warning: Current entry does not yet have a description for adding a MIME type
>>> /etc/magic, 7516: Warning: string modifier `f' invalid
>>> /etc/magic, 7517: Warning: Current entry does not yet have a description for adding a MIME type
>>> /etc/magic, 7519: Warning: string modifier `f' invalid
>>> /etc/magic, 7520: Warning: Current entry does not yet have a description for adding a MIME type
>>> /etc/magic, 7522: Warning: string modifier `f' invalid
>>> /etc/magic, 7523: Warning: Current entry does not yet have a description for adding a MIME type
>>> /etc/magic, 7525: Warning: string modifier `f' invalid
>>> /etc/magic, 7526: Warning: Current entry does not yet have a description for adding a MIME type
>>> /etc/magic, 7650: Warning: type `offset        >48' invalid
>>> /etc/magic, 7652: Warning: type `offset        <48        \b, truncated' invalid
>>> /etc/magic, 7663: Warning: Current entry does not yet have a description for adding a EXTENSION type
>>> /etc/magic, 7675: Warning: type `offset        >48' invalid
>>> /etc/magic, 7677: Warning: type `offset        <48        \b, truncated' invalid
>>> /etc/magic, 11745: Warning: Printf format is too long for type `lelong' in description `, disk signature %#-.4x'
>>> /etc/magic, 12559: Warning: Printf format is too long for type `belong' in description `,Serial %#-.8x'
>>> /etc/magic, 18311: Warning: EXTENSION type `    b_w/b&w' has bad char '&'
>>> /etc/magic, 33439: Warning: type `guid    x                %s' invalid
>>> /etc/magic, 33858: Warning: indirect offset type `q' invalid
>>> /etc/magic, 33859: Warning: indirect offset type `q' invalid
>>> /etc/magic, 33863: Warning: indirect offset type `q' invalid
>>> /etc/magic, 33871: Warning: indirect offset type `q' invalid
>>> 
>>> The new magic.mgc file was built but still no luck with recent Microsoft Powerpoint for Mac v16 files:
>>> 
>>> $ file -m /tmp/magic.mgc /tmp/CURIE_2021_PPT.pptx
>>> /tmp/CURIE_2021_PPT.pptx: Zip archive data, at least v2.0 to extract
>>> 
>>> When it should say:
>>> 
>>> $ file -m /tmp/magic.debian10.filev5.35.mgc /tmp/CURIE_2021_PPT.pptx
>>> /tmp/CURIE_2021_PPT.pptx: Microsoft PowerPoint 2007+
>>> 
>>> Any idea why?
>>> 
>>> Regards,
>>> 
>>> Frédéric.
>>> 
>>> PS : Main issue is with Amavis on RHEL8/CentOS8 considering this file as a zip file, unzipping it and then bannishing it due to .dat files considered as .exe files. :-/ Using the Debian 10 magic.mgc file on the same machine, the attachment is correctly identified as a Microsoft PowerPoint 2007+ document and the email is delivered to the recipient.
>>> 
>>> Le 01/06/2021 à 17:06, Christos Zoulas a écrit :
>>>> You need to copy the latest magic definitions into the source tree of the file version you need to use them with and build the magic.mgc file there.
>>>> If it complains about syntax, you can comment out the entries the older version of file does not like.
>>>> 
>>>> Best,
>>>> 
>>>> christos
>>>> 
>>>>> On Jun 1, 2021, at 10:58 AM, Frédéric Nass <frederic.nass at univ-lorraine.fr> wrote:
>>>>> 
>>>>> Hello,
>>>>> 
>>>>> Is it possible to use the latest magic.mgc definitions (downloaded and built from the Github repository) with an older file, lets say file v5.33 from RHEL 8 ?
>>>>> 
>>>>> I built file magic.mgc with git clone https://github.com/file/file && cd file && autoreconf -f -i && ./configure --disable-silent-rules && make -j4 but when I try to use it with file v5.33 it says:
>>>>> 
>>>>> $ file -m ./magic/magic.mgc /tmp/test.pptx
>>>>> file: File 5.33 supports only version 14 magic files. `./magic/magic.mgc' is version 16 (No such file or directory)
>>>>> 
>>>>> Is it possible to built a specific version of magic.mgc based on the latest definitions?
>>>>> 
>>>>> Best regards,
>>>>> 
>>>>> Frédéric.
>>>>> 
>>>>> --
>>>>> Cordialement,
>>>>> 
>>>>> Frédéric Nass
>>>>> 
>>>>> Direction du Numérique
>>>>> Sous-Direction Infrastructures et Services
>>>>> Université de Lorraine.
>>>>> 
>>>>> --
>>>>> File mailing list
>>>>> File at astron.com
>>>>> https://mailman.astron.com/mailman/listinfo/file
>>> 
> --
> Cordialement,
> 
> Frédéric Nass
> 
> Direction du Numérique
> Sous-Direction Infrastructures et Services
> Université de Lorraine.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 235 bytes
Desc: Message signed with OpenPGP
URL: <https://mailman.astron.com/pipermail/file/attachments/20210602/bdc5ff8a/attachment-0001.asc>

More information about the File mailing list