"/lib/ld-linux.so.2 /usr/bin/emacs" fails: "Memory exhausted"

[Mark Seaborn]

The problem can be summarised very simply:

$ /lib/ld-linux.so.2 /usr/bin/emacs
Memory exhausted--use M-x save-some-buffers RET
$ emacs --version
GNU Emacs 21.4.1
...

In other words, starting GNU Emacs by invoking the dynamic linker
doesn't work, and fails pretty quickly with an incorrect out-of-memory
error.

This is on i386.

This is the first program I've seen that fails when starting using the
dynamic linker this way. I would like this to work because Plash
(http://plash.beasts.org) launches programs this way.

The only reference to this problem that I have seen is a note in a
Linux kernel patch:

[SPARC]: Adjust 32-bit ELF_ET_DYN_BASE.

We were using 0x08000000 instead of TASK_UNMAPPED_BASE
so that running something like "/lib/ld-linux.so.2 emacs"
would work.

The issue there was that wherever /lib/ld-linux.so.2 gets
mapped (controlled by ELF_ET_DYN_BASE), that is where the
BSS start for the process ends up. Now, emacs allocates
dynamic memory for LISP objects from the BSS, and needs
the top 4 bits of the virtual address to be clear so that
it can encode LISP type and GC marking information there.

But making this obscure emacs case work breaks lots of other
stuff. For example, programs with a reasonably large data
section fail to load via direct ld.so interpreter execution
because the data section is large enough to begin overlapping
with the ELF_ET_DYN_BASE area.

The /lib/ld-linux.so.2 emacs case does not work on a lot of
platforms due to this issue, including i386, so it is not
worth making work on sparc either. It is indeed useful
sometimes when debugging a new experimental build of glibc
for example, but people doing that can hack the value of
ELF_ET_DYN_BASE in their kernels. Perhaps at some point
we will make a sysctl controllable value.

Signed-off-by: David S. Miller <***@davemloft.net>

<http://www.kernel.org/pub/linux/kernel/v2.6/snapshots/old/patch-2.6.10-rc3-bk14.log>

I don't fully understand this. Starting programs through
/lib/ld-linux.so.2 does not appear to change the addresses at which
object files get mapped.

Do you know what the problem is and whether it is simple to fix?

For comparison, XEmacs does not have this problem.

Cheers,
Mark

[Romain Francoise]

Post by Mark Seaborn
Do you know what the problem is and whether it is simple to fix?

I can reproduce this problem with Emacs 21.4 but not with the CVS
version, so it's been fixed already. It's hard to tell which change
fixed it, since the Emacs 21 and Emacs 22 codebases have diverged
significantly in the 4 years that have passed since the 21.1 release.

[Richard M. Stallman]

I don't know anything about what's happening here; I am not sure
what ld-linux.so.2 does in the first place, or whether that ought
to work, or what it ought to do.

I suggest you try with the development Emacs sources and see if the
problem still happens. If it does, I think it will be up to you to
debug it or find someone who wants to debug it. If you present an
analysis of what the problem is, maybe Emacs developers can fix it.

[Mark Seaborn]

Post by Romain Francoise

Post by Mark Seaborn
Do you know what the problem is and whether it is simple to fix?

I can reproduce this problem with Emacs 21.4 but not with the CVS
version, so it's been fixed already. It's hard to tell which change
fixed it, since the Emacs 21 and Emacs 22 codebases have diverged
significantly in the 4 years that have passed since the 21.1 release.

Thanks for checking.

It turned out that I was wrong, and the use of address space does
change when you invoke ld-linux.so.2 directly: brk() changes where it
allocates memory from. So I can fill in the gaps from the description
I posted before. brk() starts allocating from after the BSS
(zero-initialised) segment of the executable that was invoked by
exec(). For normal executables this is after 0x08000000.
ld-linux.so.2 gets loaded at 0x80000000, so brk() follows from
somewhere after that (regardless of what executable ld-linux.so.2
subsequently loads).

Emacs allocates memory using malloc(), which uses brk(), and so it
gets an address with one of the top 4 bits set, which it can't handle.

I would guess that Emacs' use of the top 4 bits hasn't changed but
rather Emacs 22 uses mmap() to allocate memory rather than malloc().

I suppose Emacs 21 could be fixed to use mmap() rather than malloc(),
though I don't know whether you'd want to make a change like that in
the stable branch. For running programs under Plash, I could change
libc's behaviour so that malloc() uses mmap() rather than brk(), which
would then work with any version of Emacs.

Mark

[Mark Seaborn]

Post by Richard M. Stallman
I don't know anything about what's happening here; I am not sure
what ld-linux.so.2 does in the first place, or whether that ought
to work, or what it ought to do.

ld-linux.so.2 is the dynamic linker on GNU/Linux:

$ /lib/ld-linux.so.2
Usage: ld.so [OPTION]... EXECUTABLE-FILE [ARGS-FOR-PROGRAM...]
You have invoked `ld.so', the helper program for shared library executables.
This program usually lives in the file `/lib/ld.so', and special directives
in executable files using ELF shared libraries tell the system's program
loader to load the helper program from this file. This helper program loads
the shared libraries needed by the program executable, prepares the program
to run, and runs it. You may invoke this helper program directly from the
command line to load and run an ELF executable file; this is like executing
that file itself, but always uses this helper program from the file you
specified, instead of the helper program file specified in the executable
file you run. This is mostly of use for maintainers to test new versions
of this helper program; chances are you did not intend to run this program.

--list list all dependencies and how they are resolved
--verify verify that given object really is a dynamically linked
object we can handle
--library-path PATH use given PATH instead of content of the environment
variable LD_LIBRARY_PATH
--inhibit-rpath LIST ignore RUNPATH and RPATH information in object names
in LIST

Plash uses a modified version of the dynamic linker, so it launches
programs by invoking ld-linux.so.2 directly. It also does this as a
technique for exec()'ing an executable from a chroot() jail without
the executable having to be inside the jail.

Mark