[Tcsh] Discard ~/.history when it grows over reasonable size

[David Kaspar [Dee'Kej]]

Hello guys,

apparently it possible for tcsh create a corrupted ~/.history file, which
can reach huge sizes. Our customer provided us with their ~/.history file
which is corrupted and causing OOM problems when running multiple instances
of tcsh simultaneously. Checking the file, I can't even open it in Vim (or
anything else actually), because it has size of 4 GiB.

My colleague dealing with this before was able to find out, that at one
point (because of this corrupted file), tcsh was trying to allocate more
than 0.5 GiB for just one line (string) from that file, which has failed
because of not sufficient memory.

This is apparently a corner case, and traditional users will most likely
never be impacted by this behaviour. However, in production environments,
this can happen much often I reckon.

So the idea that came up was to discard the ~/.history file, when it
reaches some unreasonable size like this, and especially when the size of
one line (string) is over some reasonable limit (e.g. lets say 8 KiB or
similar). I think it safe to assume that at that point the ~/.history file
can no longer serve its purpose.

Before I get into writing some patch, I would like to know if you would
agree with this proposed solution? Or do you have any better idea how to
deal with corner case?

Here's the bug report we received, if you want to check details:
https://bugzilla.redhat.com/show_bug.cgi?id=1598502

I will have to ask customer if it would be possible to share the corrupted
~/.history file in case you would like to reproduce this yourself.

EDIT: I was looking into the lexical analyzer, but I can't make much sense
of it. The only thing that I got working for me was this:

----------------------
diff --git a/sh.lex.c b/sh.lex.c
index fee790a..3160bc1 100644
--- a/sh.lex.c
+++ b/sh.lex.c
@@ -183,8 +183,14 @@ lex(struct wordent *hp)
     cleanup_ignore(hp);
     cleanup_until(hp);
     Strbuf_terminate(&histline);
+
+    if (histline.len >= INBUFSIZE) {
+        return (hadhist);
+    }
+
     if (histline.len != 0 && histline.s[histline.len - 1] == '\n')
        histline.s[histline.len - 1] = '\0';
+
     histvalid = 1;

     return (hadhist);
----------------------

With this, it was able to process the corrupted 4 GiB .history file
eventually (took several seconds), and then when exiting the tcsh, it
created "only" 138 MiB .history file. However, I guess this is not the way
you would like to fix it...

With kind regards,

David Kaspar [Dee'Kej]
*Associate Software Engineer*
*Brno, Czech Republic*

RED HAT | TRIED. TESTED. TRUSTED.
Every airline in the Fortune 500 relies on Red Hat.
Find out why at Trusted | Red Hat <http://www.redhat.com/en/about/trusted>.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.astron.com/pipermail/tcsh/attachments/20180824/6ed3fb81/attachment.html>