[GHC] #14492: Tiered memory allocation restricts available memory

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
18 messages Options
Reply | Threaded
Open this post in threaded view
|

[GHC] #14492: Tiered memory allocation restricts available memory

GHC - devs mailing list
#14492: Tiered memory allocation restricts available memory
-------------------------------------+-------------------------------------
           Reporter:  unode          |             Owner:  (none)
               Type:  bug            |            Status:  new
           Priority:  normal         |         Milestone:
          Component:  Runtime        |           Version:  8.0.2
  System                             |
           Keywords:  memory ulimit  |  Operating System:  Linux
       Architecture:  x86_64         |   Type of failure:  Poor/confusing
  (amd64)                            |  error message
          Test Case:                 |        Blocked By:
           Blocking:                 |   Related Tickets:
Differential Rev(s):                 |         Wiki Page:
-------------------------------------+-------------------------------------
 This was first noticed when running a binary produced with GHC 8.0.2
 through a queueing system (Sun Grid Engine) that limits available memory
 by setting `ulimit -v`.

 When running with **unlimited** address space, the binary reserves 1TB of
 RAM and everything works as expected.
 When running with **limited** address space, reservation happens in powers
 of 2 leading to significant unused memory at higher memory values.

 The tests are summarized in the following table:

 {{{
     Limit   VIRT    Unused  (all in GB)
     8       4       8
     10      8       2
     16      8       8
     32      16      16
     34      32      2
     64      32      32
     66      64      2
     128     64      64
     130     128     2
     256     128     128
     258     256     2
     512     256     256
     514     512     2
     1024    512     512
 }}}

 In the last line, setting `ulimit -v` to 1024GB (1073741824) causes the
 binary to only be able to reserve 512GB.
 If at runtime the software needs more than 512GB it will abort with "out
 of memory" instead of expanding to as close to the specified ulimit as
 possible.

 PS: Is there any way to make the "out of memory" error a little bit more
 informative? Something that informs the user of how much memory was
 already being used and how much more was going to be reserved when the
 failure was seen?

--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/14492>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler

_______________________________________________
ghc-tickets mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-tickets
Reply | Threaded
Open this post in threaded view
|

Re: [GHC] #14492: Tiered memory allocation restricts available memory

GHC - devs mailing list
#14492: Tiered memory allocation restricts available memory
-------------------------------------+-------------------------------------
        Reporter:  unode             |                Owner:  (none)
            Type:  bug               |               Status:  new
        Priority:  normal            |            Milestone:
       Component:  Runtime System    |              Version:  8.0.2
      Resolution:                    |             Keywords:  memory ulimit
Operating System:  Linux             |         Architecture:  x86_64
 Type of failure:  Poor/confusing    |  (amd64)
  error message                      |            Test Case:
      Blocked By:                    |             Blocking:
 Related Tickets:                    |  Differential Rev(s):
       Wiki Page:                    |
-------------------------------------+-------------------------------------

Comment (by bgamari):

 I suppose we could set an upper bound on the reservation size. That is,
 exponentially increase up to, e.g., 32GB and then from there allocate 32
 GB at a time.

--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/14492#comment:1>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler

_______________________________________________
ghc-tickets mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-tickets
Reply | Threaded
Open this post in threaded view
|

Re: [GHC] #14492: Tiered memory allocation restricts available memory

GHC - devs mailing list
In reply to this post by GHC - devs mailing list
#14492: Tiered memory allocation restricts available memory
-------------------------------------+-------------------------------------
        Reporter:  unode             |                Owner:  (none)
            Type:  bug               |               Status:  new
        Priority:  normal            |            Milestone:
       Component:  Runtime System    |              Version:  8.0.2
      Resolution:                    |             Keywords:  memory ulimit
Operating System:  Linux             |         Architecture:  x86_64
 Type of failure:  Poor/confusing    |  (amd64)
  error message                      |            Test Case:
      Blocked By:                    |             Blocking:
 Related Tickets:                    |  Differential Rev(s):
       Wiki Page:                    |
-------------------------------------+-------------------------------------

Comment (by bgamari):

 > PS: Is there any way to make the "out of memory" error a little bit more
 informative? Something that informs the user of how much memory was
 already being used and how much more was going to be reserved when the
 failure was seen?

 Probably; unfortunately there are a number of places where this error is
 thrown scattered about the runtime. This should also be fixed.

--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/14492#comment:2>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler

_______________________________________________
ghc-tickets mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-tickets
Reply | Threaded
Open this post in threaded view
|

Re: [GHC] #14492: Tiered memory allocation restricts available memory

GHC - devs mailing list
In reply to this post by GHC - devs mailing list
#14492: Tiered memory allocation restricts available memory
-------------------------------------+-------------------------------------
        Reporter:  unode             |                Owner:  (none)
            Type:  bug               |               Status:  new
        Priority:  normal            |            Milestone:
       Component:  Runtime System    |              Version:  8.0.2
      Resolution:                    |             Keywords:  memory ulimit
Operating System:  Linux             |         Architecture:  x86_64
 Type of failure:  Poor/confusing    |  (amd64)
  error message                      |            Test Case:
      Blocked By:                    |             Blocking:
 Related Tickets:                    |  Differential Rev(s):
       Wiki Page:                    |
-------------------------------------+-------------------------------------

Comment (by unode):

 Replying to [comment:1 bgamari]:
 > I suppose we could set an upper bound on the reservation size. That is,
 exponentially increase up to, e.g., 32GB and then from there allocate 32
 GB at a time.

 32GB might still be quite large of a slice. On a 64GB machine that would
 be half of the total, making the software use at most ~32GB (since 64GB
 wouldn't be reservable).

 I don't know anything about GHC's internals so take this with a grain of
 salt.
 Would it make sense to instead half the previous reservation and keep
 halfing if unsuccessful? Otherwise, on systems that rely on this `ulimit
 -v` behavior, there will always be a gap between possible memory and what
 the software is actually able to use.

 My current understanding is that a process tries to reserve memory and
 doesn't try to change this value during runtime. Is it correct to say that
 any effort (or lack thereof) at start will constrain execution? If so
 shouldn't a "reserve as much as the system allows" effort be the default?

--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/14492#comment:3>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler

_______________________________________________
ghc-tickets mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-tickets
Reply | Threaded
Open this post in threaded view
|

Re: [GHC] #14492: Tiered memory allocation restricts available memory

GHC - devs mailing list
In reply to this post by GHC - devs mailing list
#14492: Tiered memory allocation restricts available memory
-------------------------------------+-------------------------------------
        Reporter:  unode             |                Owner:  (none)
            Type:  bug               |               Status:  new
        Priority:  normal            |            Milestone:
       Component:  Runtime System    |              Version:  8.0.2
      Resolution:                    |             Keywords:  memory ulimit
Operating System:  Linux             |         Architecture:  x86_64
 Type of failure:  Poor/confusing    |  (amd64)
  error message                      |            Test Case:
      Blocked By:                    |             Blocking:
 Related Tickets:                    |  Differential Rev(s):
       Wiki Page:                    |
-------------------------------------+-------------------------------------

Comment (by luispedro):

 The current behaviour is to start at 1TB and divide by 2 until success (or
 failure if it cannot allocate at least 1MB):

 https://github.com/ghc/ghc/blob/25f36bd7ba6899be6c720292528c56bc35e0f089/rts/posix/OSMem.c#L517


 If the memory available is in between two multiples of 2, then this can
 lead to a lot of unused memory. A (not so implausible, I think our systems
 might run something similar) worst-case scenario is running on a system
 with 1T of RAM, but where the sysadmin has wisely chosen to reserve 1GB
 for a monitoring process. Now, a whole 511GB of RAM are wasted!

 Btw, that code mentions https://ghc.haskell.org/trac/ghc/ticket/10877 with
 a previous discussion on this issue.

 A simple solution is to replace `*len /= 2` by `*len -= *len/8`. The max
 number of calls to mmap() goes up from 14 to 67, but wastage is now at
 most 1/8th of the final allocated memory.

 (Another question is whether a GHC compiled programme could access >1TB of
 RAM? We now have a few 4TB machines and more on the way...)

--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/14492#comment:4>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler

_______________________________________________
ghc-tickets mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-tickets
Reply | Threaded
Open this post in threaded view
|

Re: [GHC] #14492: Tiered memory allocation restricts available memory

GHC - devs mailing list
In reply to this post by GHC - devs mailing list
#14492: Tiered memory allocation restricts available memory
-------------------------------------+-------------------------------------
        Reporter:  unode             |                Owner:  (none)
            Type:  bug               |               Status:  new
        Priority:  normal            |            Milestone:  8.4.1
       Component:  Runtime System    |              Version:  8.0.2
      Resolution:                    |             Keywords:  memory ulimit
Operating System:  Linux             |         Architecture:  x86_64
 Type of failure:  Poor/confusing    |  (amd64)
  error message                      |            Test Case:
      Blocked By:                    |             Blocking:
 Related Tickets:                    |  Differential Rev(s):
       Wiki Page:                    |
-------------------------------------+-------------------------------------
Changes (by bgamari):

 * cc: Jaffacake (added)
 * milestone:   => 8.4.1


Comment:

 > A simple solution is to replace `*len /= 2` by `*len -= *len/8`. The max
 number of calls to `mmap()` goes up from 14 to 67, but wastage is now at
 most 1/8th of the final allocated memory.

 Indeed this sounds like a pretty simple solution. I don't even think the
 increase in number of `mmap` calls is all that bad since these are failed
 `mmap`s, so they shouldn't incur the usual overhead.

 What do you think, simonmar?

--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/14492#comment:5>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler

_______________________________________________
ghc-tickets mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-tickets
Reply | Threaded
Open this post in threaded view
|

Re: [GHC] #14492: Tiered memory allocation restricts available memory

GHC - devs mailing list
In reply to this post by GHC - devs mailing list
#14492: Tiered memory allocation restricts available memory
-------------------------------------+-------------------------------------
        Reporter:  unode             |                Owner:  (none)
            Type:  bug               |               Status:  new
        Priority:  normal            |            Milestone:  8.4.1
       Component:  Runtime System    |              Version:  8.0.2
      Resolution:                    |             Keywords:  memory ulimit
Operating System:  Linux             |         Architecture:  x86_64
 Type of failure:  Poor/confusing    |  (amd64)
  error message                      |            Test Case:
      Blocked By:                    |             Blocking:
 Related Tickets:                    |  Differential Rev(s):  Phab:D4215
       Wiki Page:                    |
-------------------------------------+-------------------------------------
Changes (by bgamari):

 * differential:   => Phab:D4215


Comment:

 Here is a quick patch.

--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/14492#comment:6>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler

_______________________________________________
ghc-tickets mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-tickets
Reply | Threaded
Open this post in threaded view
|

Re: [GHC] #14492: Tiered memory allocation restricts available memory

GHC - devs mailing list
In reply to this post by GHC - devs mailing list
#14492: Tiered memory allocation restricts available memory
-------------------------------------+-------------------------------------
        Reporter:  unode             |                Owner:  (none)
            Type:  bug               |               Status:  patch
        Priority:  normal            |            Milestone:  8.4.1
       Component:  Runtime System    |              Version:  8.0.2
      Resolution:                    |             Keywords:  memory ulimit
Operating System:  Linux             |         Architecture:  x86_64
 Type of failure:  Poor/confusing    |  (amd64)
  error message                      |            Test Case:
      Blocked By:                    |             Blocking:
 Related Tickets:                    |  Differential Rev(s):  Phab:D4215
       Wiki Page:                    |
-------------------------------------+-------------------------------------
Changes (by bgamari):

 * status:  new => patch


--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/14492#comment:7>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler

_______________________________________________
ghc-tickets mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-tickets
Reply | Threaded
Open this post in threaded view
|

Re: [GHC] #14492: Tiered memory allocation restricts available memory

GHC - devs mailing list
In reply to this post by GHC - devs mailing list
#14492: Tiered memory allocation restricts available memory
-------------------------------------+-------------------------------------
        Reporter:  unode             |                Owner:  (none)
            Type:  bug               |               Status:  patch
        Priority:  normal            |            Milestone:  8.4.1
       Component:  Runtime System    |              Version:  8.0.2
      Resolution:                    |             Keywords:  memory ulimit
Operating System:  Linux             |         Architecture:  x86_64
 Type of failure:  Poor/confusing    |  (amd64)
  error message                      |            Test Case:
      Blocked By:                    |             Blocking:
 Related Tickets:                    |  Differential Rev(s):  Phab:D4215
       Wiki Page:                    |
-------------------------------------+-------------------------------------

Comment (by bgamari):

 Another option, as suggested by Jaffacake on Diff:D4215, would be to probe
 the limit. This test program
 {{{#!c
 #include <stdlib.h>
 #include <stdio.h>
 #include <sys/time.h>
 #include <sys/resource.h>

 void print_limit(const char* name, int resource) {
         struct rlimit lim;
         if (getrlimit(resource, &lim) != 0) {
                 printf("%s: uh oh\n", name);
                 exit(1);
         }
         printf("%s: cur: %d\n", name, lim.rlim_cur);
         printf("%s: max: %d\n", name, lim.rlim_max);
 }

 int main() {
         print_limit("AS", RLIMIT_AS);
         print_limit("DATA", RLIMIT_DATA);
         print_limit("RSS", RLIMIT_RSS);
         print_limit("STACK", RLIMIT_STACK);
         return 0;
 }
 }}}

 reveals that `ulimit -v` affects `RLIMIT_AS`, which can be probed with
 `getrlimit`.

--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/14492#comment:8>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler

_______________________________________________
ghc-tickets mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-tickets
Reply | Threaded
Open this post in threaded view
|

Re: [GHC] #14492: Tiered memory allocation restricts available memory

GHC - devs mailing list
In reply to this post by GHC - devs mailing list
#14492: Tiered memory allocation restricts available memory
-------------------------------------+-------------------------------------
        Reporter:  unode             |                Owner:  bgamari
            Type:  bug               |               Status:  patch
        Priority:  normal            |            Milestone:  8.6.1
       Component:  Runtime System    |              Version:  8.0.2
      Resolution:                    |             Keywords:  memory ulimit
Operating System:  Linux             |         Architecture:  x86_64
 Type of failure:  Poor/confusing    |  (amd64)
  error message                      |            Test Case:
      Blocked By:                    |             Blocking:
 Related Tickets:                    |  Differential Rev(s):  Phab:D4215
       Wiki Page:                    |
-------------------------------------+-------------------------------------
Changes (by bgamari):

 * owner:  (none) => bgamari


Comment:

 I'll try to take care of this.

--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/14492#comment:10>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler

_______________________________________________
ghc-tickets mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-tickets
Reply | Threaded
Open this post in threaded view
|

Re: [GHC] #14492: Tiered memory allocation restricts available memory

GHC - devs mailing list
In reply to this post by GHC - devs mailing list
#14492: Tiered memory allocation restricts available memory
-------------------------------------+-------------------------------------
        Reporter:  unode             |                Owner:  bgamari
            Type:  bug               |               Status:  patch
        Priority:  normal            |            Milestone:  8.6.1
       Component:  Runtime System    |              Version:  8.0.2
      Resolution:                    |             Keywords:  memory ulimit
Operating System:  Linux             |         Architecture:  x86_64
 Type of failure:  Poor/confusing    |  (amd64)
  error message                      |            Test Case:
      Blocked By:                    |             Blocking:
 Related Tickets:                    |  Differential Rev(s):  Phab:D4215
       Wiki Page:                    |
-------------------------------------+-------------------------------------

Comment (by luispedro):

 Just to ping everyone on this. We are still having to over allocate memory
 on our servers to work around this bug, while the simple 1-liner above
 (`*len -= *len/8`) would get us a 90% solution.

--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/14492#comment:11>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler

_______________________________________________
ghc-tickets mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-tickets
Reply | Threaded
Open this post in threaded view
|

Re: [GHC] #14492: Tiered memory allocation restricts available memory

GHC - devs mailing list
In reply to this post by GHC - devs mailing list
#14492: Tiered memory allocation restricts available memory
-------------------------------------+-------------------------------------
        Reporter:  unode             |                Owner:  bgamari
            Type:  bug               |               Status:  patch
        Priority:  high              |            Milestone:  8.6.1
       Component:  Runtime System    |              Version:  8.0.2
      Resolution:                    |             Keywords:  memory ulimit
Operating System:  Linux             |         Architecture:  x86_64
 Type of failure:  Poor/confusing    |  (amd64)
  error message                      |            Test Case:
      Blocked By:                    |             Blocking:
 Related Tickets:                    |  Differential Rev(s):  Phab:D4215
       Wiki Page:                    |
-------------------------------------+-------------------------------------
Changes (by bgamari):

 * priority:  normal => high


Comment:

 Right, I have a patch pending that I'll try to dust off. Thanks for the
 ping!

--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/14492#comment:12>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler

_______________________________________________
ghc-tickets mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-tickets
Reply | Threaded
Open this post in threaded view
|

Re: [GHC] #14492: Tiered memory allocation restricts available memory

GHC - devs mailing list
In reply to this post by GHC - devs mailing list
#14492: Tiered memory allocation restricts available memory
-------------------------------------+-------------------------------------
        Reporter:  unode             |                Owner:  bgamari
            Type:  bug               |               Status:  patch
        Priority:  high              |            Milestone:  8.6.1
       Component:  Runtime System    |              Version:  8.0.2
      Resolution:                    |             Keywords:  memory ulimit
Operating System:  Linux             |         Architecture:  x86_64
 Type of failure:  Poor/confusing    |  (amd64)
  error message                      |            Test Case:
      Blocked By:                    |             Blocking:
 Related Tickets:                    |  Differential Rev(s):  Phab:D4215
       Wiki Page:                    |
-------------------------------------+-------------------------------------

Comment (by Ben Gamari <ben@…>):

 In [changeset:"730781b42a003604cfa047a02280757a07b09581/ghc" 730781b/ghc]:
 {{{
 #!CommitTicketReference repository="ghc"
 revision="730781b42a003604cfa047a02280757a07b09581"
 rts/posix: Use less aggressive backoff schedule for heap reservation
 sizing

 When we allocate the heap on POSIX platforms we generally just ask for a
 1TB chunk of address space and call it a day. However, if the user has
 set a ulimit then this request will fail. In this case we would
 previously try successively smaller allocation requests, reducing the
 request size by a factor of two each time.

 However, this means that GHC will significantly allocate a significantly
 smaller heap than the available physical memory size in some
 circumstances.  Imagine, for instance, a machine with 512 GB of physical
 memory but a ulimit of 511 GB: we would be limited to a 256 GB heap.

 We now use a less aggressive back-off policy, reducing by one-eighth the
 last allocation size each try.

 Thanks to luispedro for the suggested approach.

 Test Plan: Validate

 Reviewers: simonmar, erikd

 Subscribers: rwbarton, thomie

 GHC Trac Issues: #14492

 Differential Revision: https://phabricator.haskell.org/D4215
 }}}

--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/14492#comment:13>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler

_______________________________________________
ghc-tickets mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-tickets
Reply | Threaded
Open this post in threaded view
|

Re: [GHC] #14492: Tiered memory allocation restricts available memory

GHC - devs mailing list
In reply to this post by GHC - devs mailing list
#14492: Tiered memory allocation restricts available memory
-------------------------------------+-------------------------------------
        Reporter:  unode             |                Owner:  bgamari
            Type:  bug               |               Status:  patch
        Priority:  high              |            Milestone:  8.6.1
       Component:  Runtime System    |              Version:  8.0.2
      Resolution:                    |             Keywords:  memory ulimit
Operating System:  Linux             |         Architecture:  x86_64
 Type of failure:  Poor/confusing    |  (amd64)
  error message                      |            Test Case:
      Blocked By:                    |             Blocking:
 Related Tickets:                    |  Differential Rev(s):  Phab:D4215
       Wiki Page:                    |
-------------------------------------+-------------------------------------

Comment (by unode):

 bgamari thanks for the fix.

 There's also the 1TB maximum reservation that luispedro mentioned on
 comment:4 which seems to be set on
 https://git.haskell.org/ghc.git/blob/730781b42a003604cfa047a02280757a07b09581:/rts/sm/MBlock.c#l663

 Implementing the probing solution you proposed on comment:8 would also
 address the 1TB limit.\\
 Did you manage to get anywhere with that approach?

 Thanks again.

--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/14492#comment:14>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler

_______________________________________________
ghc-tickets mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-tickets
Reply | Threaded
Open this post in threaded view
|

Re: [GHC] #14492: Tiered memory allocation restricts available memory

GHC - devs mailing list
In reply to this post by GHC - devs mailing list
#14492: Tiered memory allocation restricts available memory
-------------------------------------+-------------------------------------
        Reporter:  unode             |                Owner:  bgamari
            Type:  bug               |               Status:  patch
        Priority:  high              |            Milestone:  8.6.1
       Component:  Runtime System    |              Version:  8.0.2
      Resolution:                    |             Keywords:  memory ulimit
Operating System:  Linux             |         Architecture:  x86_64
 Type of failure:  Poor/confusing    |  (amd64)
  error message                      |            Test Case:
      Blocked By:                    |             Blocking:
 Related Tickets:                    |  Differential Rev(s):  Phab:D4215
       Wiki Page:                    |
-------------------------------------+-------------------------------------

Comment (by unode):

 PS: On your patch above if I use:

 {{{
 #!diff
 -        printf("%s: cur: %d\n", name, lim.rlim_cur);
 -        printf("%s: max: %d\n", name, lim.rlim_max);
 +        printf("%s: cur: %lu\n", name, lim.rlim_cur);
 +        printf("%s: max: %lu\n", name, lim.rlim_max);
 }}}

 I get correct values from the test program, both for AS and RSS.

--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/14492#comment:15>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler

_______________________________________________
ghc-tickets mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-tickets
Reply | Threaded
Open this post in threaded view
|

Re: [GHC] #14492: Tiered memory allocation restricts available memory

GHC - devs mailing list
In reply to this post by GHC - devs mailing list
#14492: Tiered memory allocation restricts available memory
-------------------------------------+-------------------------------------
        Reporter:  unode             |                Owner:  bgamari
            Type:  bug               |               Status:  patch
        Priority:  high              |            Milestone:  8.6.1
       Component:  Runtime System    |              Version:  8.0.2
      Resolution:                    |             Keywords:  memory ulimit
Operating System:  Linux             |         Architecture:  x86_64
 Type of failure:  Poor/confusing    |  (amd64)
  error message                      |            Test Case:
      Blocked By:                    |             Blocking:
 Related Tickets:                    |  Differential Rev(s):  Phab:D4215
       Wiki Page:                    |
-------------------------------------+-------------------------------------

Comment (by Ben Gamari <ben@…>):

 In [changeset:"26273774661bd0780b1ae8f0755ea135a0ceaf92/ghc"
 26273774/ghc]:
 {{{
 #!CommitTicketReference repository="ghc"
 revision="26273774661bd0780b1ae8f0755ea135a0ceaf92"
 rts: Query system rlimit for maximum address-space size

 When we attempt to reserve the heap, we query the system's rlimit to
 establish the starting point for our search over sizes.

 Test Plan: Validate

 Reviewers: erikd, simonmar

 Reviewed By: simonmar

 Subscribers: rwbarton, thomie, carter

 GHC Trac Issues: #14492

 Differential Revision: https://phabricator.haskell.org/D4754
 }}}

--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/14492#comment:16>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler

_______________________________________________
ghc-tickets mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-tickets
Reply | Threaded
Open this post in threaded view
|

Re: [GHC] #14492: Tiered memory allocation restricts available memory

GHC - devs mailing list
In reply to this post by GHC - devs mailing list
#14492: Tiered memory allocation restricts available memory
-------------------------------------+-------------------------------------
        Reporter:  unode             |                Owner:  bgamari
            Type:  bug               |               Status:  closed
        Priority:  high              |            Milestone:  8.6.1
       Component:  Runtime System    |              Version:  8.0.2
      Resolution:  fixed             |             Keywords:  memory ulimit
Operating System:  Linux             |         Architecture:  x86_64
 Type of failure:  Poor/confusing    |  (amd64)
  error message                      |            Test Case:
      Blocked By:                    |             Blocking:
 Related Tickets:                    |  Differential Rev(s):  Phab:D4215
       Wiki Page:                    |
-------------------------------------+-------------------------------------
Changes (by bgamari):

 * status:  patch => closed
 * resolution:   => fixed


--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/14492#comment:17>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler

_______________________________________________
ghc-tickets mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-tickets
Reply | Threaded
Open this post in threaded view
|

Re: [GHC] #14492: Tiered memory allocation restricts available memory

GHC - devs mailing list
In reply to this post by GHC - devs mailing list
#14492: Tiered memory allocation restricts available memory
-------------------------------------+-------------------------------------
        Reporter:  unode             |                Owner:  bgamari
            Type:  bug               |               Status:  closed
        Priority:  high              |            Milestone:  8.6.1
       Component:  Runtime System    |              Version:  8.0.2
      Resolution:  fixed             |             Keywords:  memory ulimit
Operating System:  Linux             |         Architecture:  x86_64
 Type of failure:  Poor/confusing    |  (amd64)
  error message                      |            Test Case:
      Blocked By:                    |             Blocking:
 Related Tickets:                    |  Differential Rev(s):  Phab:D4215
       Wiki Page:                    |
-------------------------------------+-------------------------------------

Comment (by Ben Gamari <ben@…>):

 In [changeset:"233d8150e672494dc5764d0dad5ea721a56517a1/ghc"
 233d8150/ghc]:
 {{{
 #!CommitTicketReference repository="ghc"
 revision="233d8150e672494dc5764d0dad5ea721a56517a1"
 rts: Ignore RLIMIT_AS if it is zero

 Reviewers: erikd, simonmar

 Reviewed By: simonmar

 Subscribers: rwbarton, thomie, carter

 GHC Trac Issues: #14492

 Differential Revision: https://phabricator.haskell.org/D4811
 }}}

--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/14492#comment:18>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler

_______________________________________________
ghc-tickets mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-tickets