[File] Piping into file; will it bloat RAM?

[Christos Zoulas]

Yes, plus some random constant amount of memory for internal bookkeeping.

christos

> On Oct 20, 2022, at 3:59 PM, Arif Driessen <arifd86 at gmail.com> wrote:
> 
> Hi Christos,
> 
> 
> 
> thanks for the reply. This sounds like great news! But can i just confirm my understanding, so with no changes to the default, if i pipe a 180GiB zip file into it, at max only 7M of RAM will be allocated?
> 
> 
> On Thu, 20 Oct 2022, 15:20 Christos Zoulas, <christos at zoulas.com <mailto:christos at zoulas.com>> wrote:
> 
> It will generally allocate and read only up to bytes_max which defaults to 7M and can be set by -P bytes=
> 
> 
> 
> Best,
> 
> 
> 
> christos
> 
> 
> 
>> On Oct 20, 2022, at 5:55 AM, Arif Driessen <arifd86 at gmail.com <mailto:arifd86 at gmail.com>> wrote:
>> 
>> 
>> Hello,
>> 
>> 
>> 
>> I am using `file` to get the mime-type of blobs of data streamed to me. I open a process and a pipe, and a pass the bytes as they come to me to `file`. Then I close the pipe and wait for `file` to return me a mime-type.
>> 
>> 
>> 
>> This works wonderfully, except, I'm worried, that file may be collecting into RAM the entire contents of the buffer as they are being piped.
>> 
>> 
>> For example, if I am given an 180GiB zip file, and I keep feeding those bytes to the pipe, will it blow up my RAM? Or does `file` employ something cleverl; even if just writing to the file system, or perhaps working on chunks and forgetting the past?
>> 
>> 
>> Thanks for your help!
>> 
>> 
>> Arif Driessen
>> 
>> 
>> 
>> --
>> File mailing list
>> File at astron.com <mailto:File at astron.com>
>> https://mailman.astron.com/mailman/listinfo/file <https://mailman.astron.com/mailman/listinfo/file>
>> <sanitizer.log>
>> 
> 
> 
> 
> 
> 
> 
> 
> <sanitizer.log>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.astron.com/pipermail/file/attachments/20221021/fee74dc9/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 235 bytes
Desc: Message signed with OpenPGP
URL: <https://mailman.astron.com/pipermail/file/attachments/20221021/fee74dc9/attachment.asc>