Compile And Patch Your Own Secure Linux Kernel With PaX and Grsecurity

The reason why Linux and open source will always be more secure than Windows is because you can build advanced security right in. This guide’s going to show you how to do that. I’ll be explaining how to compile your own kernel with PaX and Grsecurity patches. (I’m actually compiling 3.3.6 for myself but this guide covers 3.2.18.)

First let’s explain a few things…

What Is A Kernel?

Well, if you don’t know what a kernel is this guide is probably a bit too advanced but far be it from me to turn you away. The kernel is essentially the core of the operating system. It is the lowest level of software on the system running with the highest rights. Linux runs on “The Linux Kernel” and that kernel is common to all Linux distributions.

What Is PaX?

PaX is a series of patches that can be applied to the Linux kernel. These patches provide functionality and improvements to built in features such as ASLR (Address Space Layout Randomization), which help mitigate and deter exploits.

PaX features can be daunting so for a per-feature explanation see this link. I highly suggest you take a look at these features and research both PaX and Grsecurity so you can fully understand what they provide.

What Is Grsecurity?

Grsecurity is another series of patches that can be applied to the Linux kernel. These patches will allow changes to policy, provide a tool for RBAC, and add new policies. Grsecurity and PaX are built to work together and reinforce one another.

Why Are We Doing This?

Well, to get the most secure system that you can possibly get you’re going to want to start compiling your own kernel. The kernel is, for various reasons, where security belongs (more on this in future posts.) Now, on a default install of some distro like, say, Ubuntu you’re fairly secure and the tools already provided can be used to really lock the system down (think AppArmor) but if you’re looking to secure your system against sophisticated and targeted attacks PAX and Grsecurity are the way to go.

Keep in mind that the settings I use are for my own system. A server would be able to use more settings because it wouldn’t have to worry about, for example, the sound working.

You will also need to do this for new updates from kernel.org in order to get the latest security patches, among other things. Not even Grsecurity will keep an unpatched system safe.

So, since we’re done with the intro…

Setting Up Our System

Before we even download source code we need to make sure we have everything necessary to actually compile and configure our kernel. Let’s start by installing some software: (I’ll assume sudo/root for this guide.)

# apt-get install patch bin86 kernel-package build-essential libncurses5-dev gcc-*-plugin-dev

That’s (hopefully) everything you need to compile, package, and configure the kernel. (You may need to install gcc and plugins.)

Now let’s set up a working folder and cd to it.

# mkdir /usr/src/linux/

# cd /usr/src/linux/

That’s it for now.

Downloading The Linux Kernel And Patches

In this example I’ll be using the Linux Kernel 3.2.18 (the same steps will work for any kernel, just change the patches) and the Grsecurity patch that goes along with it. I’m assuming you’re still in /usr/src/linux/.

# wget https://grsecurity.net/stable/grsecurity-2.9-3.2.18-201205301835.patch

# wget https://www.kernel.org/pub/linux/kernel/v3.0/linux-3.2.18.tar.bz2

Once the download has completed you can extract linux-3.2.18.tar.bz2 to ./linux-3.2.18.

Patching The Kernel

This part is fairly simple, move to your new linux-3.2.18 folder and patch.

# cd ./linux-3.2.18

# patch -p1 < /usr/src/linux/grsecurity-2.9-3.2.18-201205301835.patch

This should patch the kernel without any interaction necessary.

Configuring The Kernel

(note: I am taking screenshots of a 3.3.6 kernel with my own configuration, yours may vary. Once size does not fit all – do research!)

#make menuconfig

You’ll be brought to this screen:

Note: If you want to enable KERNEXEC you must go to Processor Features and disable Xen/Paravirtualization.

Navigate (arrow keys) to Security Options and hit ‘Enter’ and…:

Configuring Grsecurity:

As you can see there are many options here. You can use preconfigured options (Low, Medium, High) or, as I suggest, Custom.

The following settings are suggestions that work for me. They can make or break systems depending on usage. A server might need completely different settings, for example. You’ll probably be compiling a few kernels just to see what breaks when.

Memory Protections:

[*] Deny reading/writing to /dev/kmem, /dev/mem, and /dev/port
[  ] Disable privileged I/O
[*] Harden ASLR against information leaks and entropy reduction
[*] Deter exploit bruteforcing
[*] Harden module auto-loading
[*] Hide kernel symbols
[*] Active kernel exploit response

Role Based Access Control (RBAC): I’ll post later about configuring RBAC.

[  ] Disable RBAC system
[*] Hide kernel processes
(3) Maximum tries before password lockout
(30) Time to wait after max password tries, in seconds

Filesystem Protections:

[*] Proc restrictions
[*] Restrict /proc to user only
[*] Additional restrictions
[*] Linking restrictions
[*] FIFO restrictions
[  ] Sysfs/debugfs restriction     -Breaks Sound on Ubuntu
[  ] Runtime read-only mount protection   -For servers, not users.
[*] Chroot jail restrictions
[*] Deny mounts
[*] Deny double-chroots

[*] Deny pivot_root in chroot

[*] Enforce chdir(“/”) on all chroots
[*] Deny (f)chmod +s
[*] Deny fchdir out of chroot
[*] Deny mknod
[*] Deny shmat() out of chroot
[*] Deny access to abstract AF_UNIX sockets out of chroot
[*] Protect outside processes
[*] Restrict priority changes
[*] Deny sysctl writes

[*] Capability restrictions

Kernel Auditing:

Your choice, this is more useful for servers. As a user I don’t need any of this.

Executable Protections:

[*] Dmesg(8) restriction
[*] Deter ptrace-based process snooping
[*] Require read access to ptrace sensitive binaries
[*] Enforce consistent multithreaded privileges
[*] Trusted Path Execution (TPE)
[*] Partially restrict all non-root users -This is one option that I use. It may break things and it’s not a critical feature so you may consider removing it.

Network Protections:

[*] Larger entropy pools
[*] TCP/UDP blackhole and LAST_ACK DoS prevention
[  ] Socket restrictions

SysCtl Support:

[  ] Sysctl support
[  ] Turn on features by default

Configuring PAX:

As with Grsecurity we’re presented with quite a few options. PAX settings can very easily break things so my settings will likely vary from what your settings could be. I will note specific areas where there are known conflicts.

For details on PaX features see this link.

PaX Control:

[  ] Support soft mode By leaving this off we have PaX system wide.
[  ] Use legacy ELF header marking
[  ] Use ELF program header marking
[  ] Use filesystem extended attributes marking

Non-Executable Pages:

[*] Enforce non-executable pages
[*] Paging based non-executable pages
[*] Emulate trampolines
[*] Restrict mprotect() This will break X.
[  ] Use legacy/compat protection demoting (read help)
[  ] Allow ELF text relocations (read help)

[*] Enforce non-executable kernel pages (Requires  you disabled Xen)
Return Address Instrumentation Method (bts) —>

Address Space Layout Randomization:

[*] Address Space Layout Randomization
[*] Randomize kernel stack base
[*] Randomize user stack base
[*] Randomize mmap() base

Miscellaneous Hardening Features: (Note: Users of ATI SPUs must use my settings here. Users with other GPUs should enable all ** settings.)

[*] Sanitize all freed memory
[*] Sanitize kernel stack
[*] Prevent various kernel object reference counter overflows  **
[*] Harden heap object copies between kernel and userland  **
[*] Prevent various integer overflows in function size parameters

**As of version 12.8 the closed source ATI FGLRX drivers are compatible with the above. Earlier versions will NOT work.

Other Security Settings

Exit your PaX options and go back to the security settings.

Set ” Low address space for LSM to protect from user allocation” to 65536 or 32768:

This is the portion of low virtual memory which should be protected
from userspace allocation. Keeping a user from writing to low pages
can help reduce the impact of kernel NULL pointer bugs.

For most ia64, ppc64 and x86 users with lots of address space
a value of 65536 is reasonable and should cause no problems.
On arm and other archs it should not be higher than 32768.
Programs which use vm86 functionality or have some need to map
this low address space will need the permission specific to the
systems running LSM.

Time To Compile And Install

(Note: You may want to see the last section ‘Other Tips’ before you compile.)

# make-kpkg clean

# make-kpkg --initrd --append-to-version "grsec1.0" kernel_image

The above command might give you an error in which case you’ll need to type it out. It’s a wordpress issue where it combines the –‘s.

Compiling will take a while. Probably a half hour, potentially multiple hours depending on the hardware. When you’re done there will be a package in /usr/src/linux.

# cd /usr/src/linux

# dpkg -i ./<name.of.deb>

Reboot into the new kernel and that’s it.

It’s a lot of work to set it up but it should only take you a few minutes after the first time. Compiling is what takes the longest but it can be done in the background.

If something breaks just boot into the last kernel you used. Simple.

Other Tips

Since you’re taking the time to compile this thing you may as well get the most out of it. I suggest you go through the settings and disable any drivers not related/ enable or disable features.

You should also open /usr/src/linux/linux-3.2.18/makefile and find the line:

# Add user supplied CPPFLAGS, AFLAGS and CFLAGS as the last assignments

And have the following line as: (add the -march=native -pipe -mtune=native)

KBUILD_CPPFLAGS += $(KCPPFLAGS) -march=native -pipe -mtune=native

pipe will speed up compile time significantly for systems with lots of RAM. march=native will optimize the kernel for your specific CPU. Don’t use -march=native if you are compiling for another system with a different CPU (same for mtune).

Take the time to disable drivers that you won’t use. It’ll speed up compile time and produce a smaller kernel.

Highlights

The highlights of this new kernel are:

1) Improved ASLR

2) RBAC

3) Hardened Chroots (if you use Chromium or Google Chrome the sandbox makes use of Chroots so you’ve just made it more secure.)

4) Exploit bruteforcing is much harder

5) A generally more secure system with stricter policies and checks

6) Optimized kernel if you followed the “notes.”

This was a long post so if I’ve left anything out or misrepresented something leave a comment and let me know. RBAC is worth its own full post so I’ll get to that later (setting it up on any user-oriented OS is difficult.) For now you can see this wiki page to learn how to set RBAC up, but it’s not so simple.

Sources:

http://www.howtoforge.com/hardening-the-linux-kernel-with-grsecurity-debian

http://www.gentoo.org/proj/en/hardened/pax-quickstart.xml

https://en.wikibooks.org/wiki/Grsecurity/Print_version

51 thoughts on “Compile And Patch Your Own Secure Linux Kernel With PaX and Grsecurity

  1. Good point, the only thing I’d ask is why disable TPE? It’s good in case someone manages to upload an exploit to your webserver. If it’s user nobody, they won’t be able to execute it.

  2. It would be great if you could setup a repository and release a 3.2 series grsecurity image targeted at the desktop. The Tails project could really use it. Tails uses a program called Tor to let its users communicate anonymously. Activists and others use this who are targeted.

    • I could do this, yes. The issue is really just that I’d have to compile twice as I compile to optimize for my specific architecture. I’ve thought about it before though.

  3. >>
    [ ] Sysfs/debugfs restriction -Breaks Sound on Ubuntu
    [ ] Runtime read-only mount protection -For servers, not users.
    >>

    Sysfs/debugfs restriction breaks only pulse audio, alsa works.
    Runtime read-only mount protection is not only handy for servers but also for desktop/workstations where you want the kernel to obey /etc/fstab /etc/mtab ro(read-only) settings (enforce).

    • I think it’s a rare situation where a user wants read-only mount protection like what Grsecurity enforces. Naturally different situations call for different security measures but typical users won’t want or need this.

      Yep, it’s just a PulseAudio issue. I can specify that in the guide I suppose.

  4. Thanks for a great article.
    You have mention that you are compiling the grsecurity for Linux 3.3.6.
    I have version 3.3.
    Can you please write if there are changes, I will really appreciate if you patch the changes.
    Thank you
    Jons

  5. Pingback: Explaining Chrome's Linux Sandbox » InsanityBit InsanityBit

  6. Pingback: Grsecurity On Desktop Is Simple - InsanityBit

  7. Pingback: Hardening DNSCrypt - InsanityBit

  8. Using a mainline kernel as a base is not a great idea.

    You see, Ubuntu carries quite some patches on top of mainline kernels and does so for good reasons. Their backports mostly deal with driver stability (most notably GPU drivers) but they also carry patches like AuFS and OverlayFS without which you can’t make a LiveCD.

    Ubuntu desktop kernel configuration is also carried as a series of patches (see https://wiki.ubuntu.com/Kernel/FAQ/UbuntuDelta). Which means you’ll have to configure the absolutely generic kernel for yourself by hand. Definitely NOT a great idea!
    And essential Ubuntu packaging is not present in mainline kernels either, so I have some doubts regarding correctness of generated .deb. I’m pretty sure it breaks dkms, for example.

    This can be mitigated by using Ubuntu kernels as a base, but I’m not sure if the patches will still apply cleanly.

    But either way you have to manually monitor the mainline kernel for security updates, not to mention manually re-downloading, re-patching and re-building the kernel when an update is issued.
    Better mention it in the article – I’ve learned the hard way that people are extremely fond of ‘install and forget’ approach to software, even kernels, completely ignoring the possibility of security updates.

    • If you’re using the latest kernel you still get security patches. You lose some of the Ubuntu specific things, but none of it effects security, and your system still boots, you just loose a few features. I will note this.

      • True. I just wanted to point out that you have to care about security updates to a custom-built kernel by yourself, you no longer get them automatically with the rest of system updates.
        I’m sure it’s obvious to you but I’ve seen so many people just installing .deb’s for mainline kernels and forgetting about them, that I feel it should be explicitly noted in the guide.

      • Can you expand on what Ubuntu specific features would be lost when using the mainline kernel?

        I’m following your guide and patching the mainline kernel with grsec and I’d like to use it on my Ubuntu machine.

        • Nothing major, as far as I know. Some drivers may be missing. One example is that before Ubuntu used the 3.5 kernel they backported Seccomp Mode 2 Filter support to their stable kernel, but I don’t know anything critical that’s missing.

          • Ubuntu has three AppArmor patches (https://launchpad.net/apparmor) for 3.2.*

            0001-AppArmor-compatibility-patch-for-v5-network-controll.patch
            0002-AppArmor-compatibility-patch-for-v5-interface.patch
            0003-AppArmor-Allow-dfa-backward-compatibility-with-broke.patch

            I’ve had trouble getting AA to work on a mainline kernel, but I think grsec includes them by default.

  9. Would I be correct that this can not be used at the moment due to GrSecurity patch file being out dated with the most recent kernel 3.10.6?
    because 3.10.5 is in testing and well its already outdated lol.
    despite their being minor differences to the kernel itself from 3.10.5 – 3.10.6.
    Also anyone using the test version I.E. beta had any severe bugs with it?
    at this point i feel its better to wait.
    thanks for the tutorials on your site though i learn something new every once in awhile like DNS crypt hardening awesome.
    Guess for now its just Apparmor.

    • just wanted to add a note as of yesterday the kernel has been updated to 3.10.7 now lol.
      so GrSecurity is really behind at the moment.

    • Once in a while they fall a patch (or two) behind, but usually it’s out within a day or two. I’m currently using 3.10.5, just waiting for the next release. I don’t really mind if they go a bit behind, because even if there were a security release in those versions, it would likely take much longer to attack a Grsecurity kernel, long enough for a patch to come out.

      In terms of severe bugs, the only one I can think of is that PAGE_KERNEXEC breaks ‘update-grub’, so I always keep a vanilla kernel installed, otherwise updating the kernel becomes impossible.

      Glad you enjoy the site.

      • After compiling kernel with grsec and pax. The update grub is getting killed. Can you please let me know how to fix this error?

        ~# update-grub
        Killed

        • Yes, you will have to disable PageExec on the grub binaries using paxctl.

          Something like paxctl -ce /usr/bin/*grub*

          It may be in /sbin though.

  10. Unfortunately, it did not work. Is there any option in the menuconfig which is causing this and if we recompile by disabling that option and it works ?

    • You could disable pageexec. It’s just a matter of disabling the option on a single binary though. Something like grub-pc, I believe. Or update-grub. I would recommend that path.

  11. In menuconfig I’ve noticed a few new options:

    Filesystem protections –>
    Kernel-enforced SymlinksIfOwnerMatch (http://en.wikibooks.org/wiki/Grsecurity/Appendix/Grsecurity_and_PaX_Configuration_Options#Kernel-enforced_SymlinksIfOwnerMatch)

    Eliminate stat/notify-based device sidechannels (http://en.wikibooks.org/wiki/Grsecurity/Appendix/Grsecurity_and_PaX_Configuration_Options#Eliminate_stat.2Fnotify-based_device_sidechannels)

    Exempt initrd tasks from restrictions (http://en.wikibooks.org/wiki/Grsecurity/Appendix/Grsecurity_and_PaX_Configuration_Options#Exempt_initrd_tasks_from_restrictions)

    I’ve personally disabled the first two as (if I understand correctly) I don’t run an Apache server and use search indexing (ie Recoll)

    I didn’t check chroot Capability restrictions; I saw Firefox using modprobe so perhaps Chrome would too (I could be completely off here)

    Also:

    Network restrictions –>
    Disable TCP simultaneous connect (http://en.wikibooks.org/wiki/Grsecurity/Appendix/Grsecurity_and_PaX_Configuration_Options#Disable_TCP_Simultaneous_Connect)

    And a few extra PaX toughies.

    Compiling broke early because of “gcc plugins”, so I had to install gcc-4.6-plugin-dev (I have gcc 4.6.3). Fingers crossed. 🙂

  12. Pingback: Ubuntu Server Hardening | Computer Ramblings

  13. Hey, I followed your guide to the letter. However, I noticed two things – the first is that even though I made sure that the Kernel and patch versions were an exact match, I got “Reversed (or previously applied) patch detected! Assume -R? [n]” a huge number of times. After I got past that and entered the screen for GRsecurity, I didn’t see any of the options that you listed above.

  14. Pingback: Make error while compiling kernel

  15. Just a couple of side comment about the tips for compiling section.

    The line
    KBUILD_CPPFLAGS += $(KCPPFLAGS) -march=native -pipe -mtune=native
    gave me a lot of trouble.

    For some reason, -march=native triggers an error that causes the compilation under gcc version 4.8.2 (from Ubuntu) to abort with my cpu.

    (The -pipe and -mtune=native arguments, separately and in combination, work fine, however.)
    This is not really an issue related to grsecurity, of course, but I mention it in case there are other inexperienced readers who are following the guide here and also experiencing problems. It’s taken me a week working through different options to figure out what was causing the error. I initially thought I had made a bad choice in the grsecurity configuration, but I found that I can’t even get the kernel to compile by itself without the grsecurity patch included.

    Other thought that probably will actually be more useful for others —
    if you set the CONCURRENCY_LEVEL during the make-kpkg command to take advantage of multicore cpus, it can save a lot of time.
    For example, while -pipe added to the Makefile shaves a mere 2 seconds off the 30 minutes it takes to compile the vanilla kernel without the grsecurity patch, adding the CONCURRENCY_LEVEL (with -j9, for an 8-core cpu, where the j value is set to be your number of cpu cores + 1) cut the compilation time by almost 70%:

    time make-kpkg --initrd kernel_image kernel_headers
    27 minutes 6 seconds

    time make-kpkg --initrd -j9 kernel_image kernel_headers
    8 minutes 32 seconds

    (I haven’t tried that with the grsecurity patch included yet though.)

    • Using -j9 to compile with the grsecurity patch and settins applied:
      9 minutes 43 seconds

      (I should mention that these numbers are all with 16gb RAM and a fast hard drive)

  16. Looking at the Gentoo Wiki article “PaX Quickstart”, in the section on kernel configuration, I noticed that it says to set PAX_LATENT_ENTROPY=y

    That is, in menuconfig:

    Code:
    PaX —>
    Miscellaneous hardening features —>
    […]
    [*] Generate some entropy during boot

    The menuconfig help description for PAX_LATENT_ENTROPY reads as follows:

    “By saying Y here the kernel will instrument some kernel code to extract some entropy from both original and artificially created program state. This will help especially embedded systems where there is little ‘natural’ source of entropy normally. The cost is some slowdown of the boot process and fork and irq processing. When pax_extra_latent_entropy is passed on the kernel command line, entropy will be extracted from up to the first 4GB of RAM while the runtime memory allocator is being initialized. This costs even more slowdown of the boot process. Note that the implementation requires a gcc with plugin support, i.e., gcc 4.5 or newer. You may need to install the supporting headers explicitly in addition to the normal gcc package. ***Note that entropy extracted this way is not cryptographically secure!***”

    … but then there is this, from the grsecurity website:

    “Providing enough randomness, especially at early boot, has been a constant struggle for Linux. While on the Intel platform, Ivy Bridge and newer processor models that support RDRAND generally eliminate the problem of entropy starvation at early boot and runtime, older processors and those of other architectures continue to be plagued by this problem. The demonstrable effects of this include weak private keys on embedded systems, weak randomness on VM guests, predictable stack canaries, and more. PaX’s latent entropy plugin included in grsecurity addresses the lack of randomness by modifying the instruction streams of functions called only during boot as well as some specific functions at runtime that perform some unpredictable computations from the perspective of an attacker. These functions are modified to produce a nondeterministic value based on the flow of execution. ***At several points in the boot process and at runtime, this pseudorandom value is mixed in to the entropy pool.*** Further, a kernel compiled with this plugin will have the various random pools in the kernel initialized at compile-time to random contents.”

    … So the configuration help says that it’s not cryptographically secure, but then their website documentation says that it’s being mixed into the entropy pool. Can you explain if this going to somehow weaken security for cryptographic applications that require random data from the entropy pool?

    Thanks so much for writing such an informative article!

    • This won’t weaken security, it just means that while this entropy is suitable for something like ASLR or stack cookies, it is not suitable for making cryptographic keys.

      This is important because the kernel uses stack cookies, and because the kernel boots quite early in the process, these stack cookies are ineffective without Grsecurity.

      • Thanks, that clears things up, I think. So basically this entropy is only being created/used during the early boot process and then later the normal entropy generation process would be the only one used, correct?

        • Yeah, basically. When the system boots up more devices it can start gathering more entropy from new sources, and that can be cryptographically secure. But right at the beginning, when almost nothing is booted up, it still needs entropy but for non-cryptographic things.

Leave a Reply

Your email address will not be published. Required fields are marked *