[File] [PATCH] Magdir/windows Microsoft Event Trace Logs *.ETL + EVTX
Christos Zoulas
christos at zoulas.com
Fri Oct 7 18:28:53 UTC 2022
I don't remember replying, but committed, thanks!
christos
> On Oct 2, 2022, at 8:23 PM, Jörg Jenderek <joerg.jen.der.ek at gmx.net> wrote:
>
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Hello,
>
> Some days ago i run the cleaning tool czkawka found on
> https://qarmin.github.io/czkawka/. One menu item concerns bad
> extensions. After running tool i looked in saved file list
> results_bad_extensions.txt for bad extension examples.
> One listed extension is ETL.
>
> These file are Microsoft Event Trace Logs found on Windows systems at
> different locations. When running file command version 5.43 on such
> ETL examples and related files i get an output like:
>
> NPETraceSession.etl: Apple HFS/HFS+ resource fork
> OpenSSH%4Admin.evtx: MS Windows Vista Event Log,
> 1 chunks (no. 0 in use), empty, DIRTY
> RestoreUI.3.etl: data
> Wifi.etl: dBase III DBT, version number 0,
> next free block index 81920, 1st item
> "10586.494.amd64fre.th2_release_
> WindowsBackup.4.etl: GeoSwath RDF
> boot_BASE+CSWITCH_1.etl: dBase III DBT, version number 0,
> next free block index 1048576, 1st item
> "\377\377\377\377\377\377\377\377\377\377
> dumpfile.evtx: MS Windows Vista Event Log,
> 7 chunks (no. 6 in use), next record no. 286
> service.0.etl: data
>
>
> Furthermore for ETL samples wrong mime type is shown with -i option.
> With option --extension 3 byte sequence etl is not shown.
>
> For comparison reason i run the file format identification utility
> TrID ( See https://mark0.net/soft-trid-e.html). This describes many
> examples (690 of 758) correctly as "Window tracing/diagnostic binary
> log" by definition etl.trid.xml (See appended trid-etl-v.txt.gz).
>
> For comparison reason i also run the file format identification
> utility DROID ( See https://sourceforge.net/projects/droid/). This
> does not recognize the ETL samples.
>
> That these are Trace Logs files can be verified by commands like:
> tracerpt.EXE AutoLogger-Diagtrack-Listener.etl -of EVTX
> etl2xml --input AMSITrace.etl --output AMSITrace.xml
>
> Luckily i found some unofficial file format information. So that
> informations are now expressed by comment lines after EVTX
> section inside Magdir/windows like:
>
> # URL: http://fileformats.archiveteam.org/wiki/ETL
> # Reference: http://mark0.net/download/triddefs_xml.7z
> # defs/e/etl.trid.xml
> # https://www.geoffchappell.com/studies/windows/
> # km/ntoskrnl/api/etw/tracelog/trace_logfile_header.htm
>
> Unfortunately ETL have no strong magic at the beginning. So i put
> displaying part inside sub routine trace-etl, This looks like:
> 0 name trace-etl
>> 0 ubyte x Windows Event Trace Log
> !:mime application/etl
> !:ext etl
>> 0 search/0x2b4/sb :\0\x5c\0
>>> &-2 lestring16 x "%s"
>
> According to unofficial Software Analyst Geoff Chappell after the
> trace log file header the names of the logger and the log file are
> stored as null-terminated Unicode strings. By last line the original
> log file name (like "c:\Windows\Logs\NetSetup\service.0.etl"
> "C:\Windows\System32\LogFiles\WMI\Wifi.etl") is shown. The header and
> the sizes are different from variant (32- or 64-bit) and also from
> version to version. So i use a brute force way to find the beginning
> by looking for DOS drive letter part ( string :\ encoded as UTF-16).
> For example boot_BASE+CSWITCH_1.etl this does not work, because the
> real log file name appears later (about 1441610h) and other ETL
> names appear before (about 699080h).
>
> According to website nirsoft.net it has an own mime type.
> That is what i found on my systems after i have installed the Windows
> Toolkit. Then ETL files are connected via registry as "wpa.etl_file"
> with Windows Performance Analyzer wpa.exe.
> But unfortunately it is not officially registered as IANA. Oh,
> Microsoft now says they become friends of open source, wants to
> improve user experience and protect the user by things like secure
> boot. This is like illness. They try to care about the symptoms and
> not the basic reasons. That is not surprising, when there exist many
> places inside the windows system, which are not explained and
> documented. So they must make high efforts to protect directories
> by their virus defender software. This would not be necessary if
> things are well documented. Remember, Microsoft is not a well fare
> society. It is a company to make money and in the past the used
> methods like mafia. So why they should have changed their behaviour
> now. So you can not trust them like Putin. The gifts they offered
> maybe under Tolkien motto "a system to find, bind and bring them
> all in the darkness to becomes slaves". Even when Microsoft would
> promise to be not evil. This does not help when an American
> government like Trump arises again and forces Microsoft as American
> company by law to give all data to him. The counter part in my eyes
> is Linus Torvalds. He has never tried to make "big" business. From
> the beginning he shared his ideas with all people. So become
> prepared that you can switch to alternatives when things become worse
> .
>
> Translating the TrID definitions the test for detections starts like:
> 0 ubyte 0
>> 0 search/0x699086/b .\0e\0t\0l\0\0\0
>>> 0 use trace-etl
> According to Geoff Chappell every ETL file begins with a
> WMI_BUFFER_HEADER, a SYSTEM_TRACE_HEADER and a TRACE_LOGFILE_HEADER.
> This seems to start with a nil byte, but i do not know if this is
> always true. After the trace log file header the names of the logger
> and the log file are stored as null-terminated Unicode strings. So
> ETL files contain near the beginning the bytes sequence
> .\0e\0\t\0l\0\0\0 triggered by used etl file name suffix encoded as
> UTF-16. Normally this search range is only some hundreds bytes, but
> for example boot_BASE+CSWITCH_1.etl i must raise this range up
> 699086h. Unfortunately this only works if in src/file.h
> FILE_BYTES_MAX is raised above 699086h (6,59 MiB).
>
> There exist a section for Vista Event Log (*.EVTX) inside
> Magdir/windows which are related with ETL files. The mentioned URL
> does not exist any more. So i replace it with archived version and
> add also site with more informations. So this now becomes like:
> # URL: https://github.com/libyal/libevtx/blob/main/
> # documentation/
> # Windows%20XML%20Event%20Log%20(EVTX).asciidoc
> # Reference: https://web.archive.org/web/20110803085000/
> # https://computer.forensikblog.de/en
> # /2007/05/some_magic.html
> # http://mark0.net/download/triddefs_xml.7z
> # defs/e/evtx.trid.xml
>
> The description happens by lines starting like:
> 0 string ElfFile\0 MS Windows Vista Event Log
>> 0x2a leshort x \b, %d chunks
> This now becomes like:
> 0 string ElfFile\0 MS Windows
> !:mime application/x-ms-evtx
> !:ext evtx
>> 0x24 ulelong =0x00030001 Vista-8.1 Event Log
>> 0x24 ulelong !0x00030001 10-11 Event Log, version
>>> 0x26 uleshort x %u
>>> 0x24 uleshort x \b.%u
>> 0x2a leshort x \b, %d chunks
> Instead of generic application/octet-stream mime type i show an user
> defined one. According to the documentation for Vista and later the
> version is 3.1 and for Windows 10 (2004) and later the version is
> 3.2. Because it is irritating when on newer Windows system files are
> described as "Vista Event Log". So for the first case show phrase
> "Vista-8.1" instead "Vista" and for newer Windows systems show phrase
> "10-11" instead "Vista" and also the version number.
>
> After applying the above mentioned modifications by patch
> file-5.43-windows-etl.diff then most ETL samples are now described
> correctly (depending on FILE_BYTES_MAX ins src/file.h). 17 examples
> are not recognized. But these seem to be only two examples
> (EtwRTEventlog-Security.etl EtwRTDiagLog.etl) in different
> directories. In found these examples on Vista system. All these
> examples start with string rlfs at the beginning and contains no
> readable file name. So i do not know if this triggered by accident or
> if this another "undocumented" feature of Microsoft. This with option
> - -m Magdir/windows now looks like:
>
> NPETraceSession.etl: Windows Event Trace Log
> "C:\Dokumente und Einstellungen\user_1234\
> Lokale Einstellungen\Anwendungsdaten\
> NPE\NPETraceInProgress.etl"
> OpenSSH%4Admin.evtx: MS Windows 10-11 Event Log, version 3.2,
> 1 chunks (no. 0 in use), empty, DIRTY
> RestoreUI.3.etl: Windows Event Trace Log
> "C:\WINDOWS\Logs\SystemRestore\
> RestoreUI.0.etl"
> Wifi.etl: Windows Event Trace Log
> "C:\Windows\System32\LogFiles\WMI\Wifi.etl"
> WindowsBackup.4.etl: Windows Event Trace Log
> "C:\Windows\Logs\WindowsBackup\
> WindowsBackup.0.1.etl"
> boot_BASE+CSWITCH_1.etl: Windows Event Trace Log
> dumpfile.evtx: MS Windows Vista-8.1 Event Log, version 3.1,
> 7 chunks (no. 6 in use), next record no. 286
> service.0.etl: Windows Event Trace Log
> "C:\Windows\Logs\NetSetup\service.0.etl"
>
> I hope my diff file can be applied in future version of file
> utility.
>
> With best wishes,
> Jörg Jenderek
> - --
> Jörg Jenderek
> -----BEGIN PGP SIGNATURE-----
> Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/
>
> iF0EARECAB0WIQS5/qNWKD4ASGOJGL+v8rHJQhrU1gUCYzorYAAKCRCv8rHJQhrU
> 1vYdAKCs0mgoxnFO8vFKxWkFvYC4q6KqjgCeJosNM8c8vwzmYynhW2n8C6o2qDg=
> =hHdi
> -----END PGP SIGNATURE-----
> <file-5_43-windows-etl_diff.DEFANGED-20826><file-5_43-windows-etl_diff_sig.DEFANGED-20827><trid-v-etl.txt.gz>--
> File mailing list
> File at astron.com
> https://mailman.astron.com/mailman/listinfo/file
> <sanitizer.log>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 235 bytes
Desc: Message signed with OpenPGP
URL: <https://mailman.astron.com/pipermail/file/attachments/20221007/5e78574c/attachment.asc>
More information about the File
mailing list