Hardening pkgsrc

Securing packages, 17.000 at a time

Pierre Pronchery

IT-Security Consultant
Defora Networks
Revision History
Revision 0.12017-01-12Pierre
Pronchery
Initial submission to AsiaBSDCon

Abstract

pkgsrc is a package management system, developed as a community project and managed by the NetBSD Foundation. Its repository is the official source of third-party software packages on the NetBSD Operating System, but its portability and overall quality has allowed it to be adopted on other platforms as well. It is notably used in industrial and scientific contexts by major companies and organizations. Proper security management of the project is therefore critical, and existing managerial and technical processes are first summarized in this paper. A number of features helping with the security hardening of binary packages have been introduced in the past few months, and are described here. Additional mechanisms still under development are presented as well, with the hope of seeing them adopted and improving the overall security level further.


Table of Contents

1 Introduction
2 Security management
2.1 Vulnerability assessment database
2.2 Security patches to the latest stable release
3 Hardening features
3.1 Package signatures
3.2 Stack Smashing Protection (SSP)
3.3 Fortify
3.4 Position-Independent Executables (PIE)
3.5 RELRO and BIND_NOW
4 Future work
4.1 Reproducible builds
4.2 Code Flow Integrity

1 Introduction

pkgsrc is a package management system, providing over 17.000 packages today [1]. Even though it originates from the NetBSD Project[2], pkgsrc supports many other platforms[3], as the official source for packages for some. This is notably the case for Joyent's SmartOS cloud computing platform[4], or can be observed at computational clusters from the NASA space agency[5].

The pkgsrc project has therefore become a critical part of the infrastructure for industrial and scientific purposes, alongside being used by a significant amount of users. As expected, the project takes particular care of the security of the software it provides. This is implemented in two ways:

  • The enforcement of management processes, aiming at tracking the security record of the infrastructure and its packages, as well as reacting to security incidents.

  • The introduction of technical counter-measures, with the goal to mitigate or thwart potential attacks.

This paper presents a summary of the procedures in place and respective teams in charge of maintaining the project in terms of security. Regardless of the presence and efficiency of the work performed by these teams, updates may not reach the users of the project in a timely manner. In some cases, the users may not be aware of the updates, or may find themselves in a context where running software cannot be modified once certified or deployed to production.

However, system hardening can help mitigating security issues as they occur. A key feature of pkgsrc is its ability to abstract the specificities of the host Operating System and build environment away. This versatile, centralized software repository provides a great opportunity to apply, experiment with and maintain security features for a complete software distribution. As a result, the architecture of pkgsrc allows mitigations to be implemented for a whole range of packages at once.

While some features and mechanisms mentioned are well adopted in the industry, their integration into the pkgsrc project is still in progress [6]. This paper describes a number of them, either readily introduced or currently under development. Their respective challenges, known impact, and future work is detailed where relevant. In the future, thanks to its particular architecture, the pkgsrc project should also be able to help research on additional innovative techniques, thereby further hardening the deployment of pkgsrc for its supported platforms and users, or possibly also for other software distributions.

2 Security management

The pkgsrc project has two distinct teams handling security aspects of the project: the pkgsrc Security Team [7] and the pkgsrc Release Engineering Group [8]. While the former is mainly in charge of tracking security issues, the latter actually updates the stable releases, notably for security issues.

2.1 Vulnerability assessment database

The Security Team for the pkgsrc project maintains a list of known vulnerabilities [9], together with a number of tools assisting the user in corresponding decisions. The list itself is assembled from different sources:

  • release notes from software developers,

  • Security Advisories from security vendors, such as Secunia[10],

  • announcements on public mailing-lists,

  • erratas and advisories from further software distributions, or governmental and technical organizations.

Additionally, the list is cryptologically signed, using PGP[11]. With the underlying systems configured to effectively refresh this database at regular intervals, the administrative users are typically sent daily reports over e-mail for known vulnerabilities in the software installed[12]. On NetBSD, this is enabled by adding a line containing fetch_pkg_vulnerabilities=YES in the /etc/daily.conf configuration file.

Alerts can be issued while building a vulnerable package from source, as illustrated in Example 1, “Installing a vulnerable package from source”. This behaviour is controlled in mk.conf[13].

Example 1. Installing a vulnerable package from source

$ make install
=> Bootstrap dependency digest>=20010302: found digest-20160304
===> Checking for vulnerabilities in xenkernel45-4.5.5nb1
Package xenkernel45-4.5.5nb1 has a information-leak vulnerability, see http://xenbits.xen.org/xsa/advisory-200.html
Package xenkernel45-4.5.5nb1 has a denial-of-service vulnerability, see http://xenbits.xen.org/xsa/advisory-203.html
Package xenkernel45-4.5.5nb1 has a denial-of-service vulnerability, see http://xenbits.xen.org/xsa/advisory-202.html
Package xenkernel45-4.5.5nb1 has a privilege-elevation vulnerability, see http://xenbits.xen.org/xsa/advisory-204.html
Package xenkernel45-4.5.5nb1 has a privilege-elevation vulnerability, see http://xenbits.xen.org/xsa/advisory-192.html
Package xenkernel45-4.5.5nb1 has a denial-of-service vulnerability, see http://xenbits.xen.org/xsa/advisory-193.html
Package xenkernel45-4.5.5nb1 has a arbitrary-code-execution vulnerability, see http://xenbits.xen.org/xsa/advisory-195.html
Package xenkernel45-4.5.5nb1 has a privilege-elevation vulnerability, see http://xenbits.xen.org/xsa/advisory-197.html
Package xenkernel45-4.5.5nb1 has a privilege-elevation vulnerability, see http://xenbits.xen.org/xsa/advisory-191.html
ERROR: Define ALLOW_VULNERABLE_PACKAGES in mk.conf or IGNORE_URL in pkg_install.conf(5) if this package is absolutely essential.
*** Error code 1

Similarly, alerts can be issued when installing a vulnerable binary package, as illustrated in Example 2, “Installing a vulnerable binary package”. In this case, behaviour is controlled in pkg_install.conf[14].

Example 2. Installing a vulnerable binary package

# pkg_add wireshark-2.2.1.tgz
Package wireshark-2.2.1 has a denial-of-service vulnerability, see https://www.wireshark.org/security/wnpa-sec-2016-58.html
Package wireshark-2.2.1 has a denial-of-service vulnerability, see https://www.wireshark.org/security/wnpa-sec-2016-59.html
Package wireshark-2.2.1 has a denial-of-service vulnerability, see https://www.wireshark.org/security/wnpa-sec-2016-60.html
Package wireshark-2.2.1 has a denial-of-service vulnerability, see https://www.wireshark.org/security/wnpa-sec-2016-61.html
Package wireshark-2.2.1 has a denial-of-service vulnerability, see https://www.wireshark.org/security/wnpa-sec-2016-62.html
Package wireshark-2.2.1 has a denial-of-service vulnerability, see https://www.wireshark.org/security/wnpa-sec-2017-01.html
Package wireshark-2.2.1 has a denial-of-service vulnerability, see https://www.wireshark.org/security/wnpa-sec-2017-02.html
pkg_add: 1 package addition failed

Instant snapshots can otherwise be requested with the pkg_admin tool, as illustrated in Figure 1, “Sample output from the pkg_admin tool”.

Figure 1. Sample output from the pkg_admin tool

# pkg_admin audit
Package ffmpeg2-2.8.6nb2 has a denial-of-service vulnerability, see https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2016-6671
Package ffmpeg2-2.8.6nb2 has a denial-of-service vulnerability, see https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2016-6881
Package ffmpeg2-2.8.6nb2 has a multiple-vulnerabilities vulnerability, see http://www.openwall.com/lists/oss-security/2016/10/08/1
Package ffmpeg2-2.8.6nb2 has a denial-of-service vulnerability, see https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2016-8595
Package ffmpeg2-2.8.6nb2 has a integer-overflow vulnerability, see http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2016-6164
Package ffmpeg1-1.2.12nb3 has a denial-of-service vulnerability, see http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2015-5479
Package ffmpeg1-1.2.12nb3 has a denial-of-service vulnerability, see https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2016-6881
Package ffmpeg1-1.2.12nb3 has a multiple-vulnerabilities vulnerability, see http://www.openwall.com/lists/oss-security/2016/10/08/1

2.2 Security patches to the latest stable release

Security updates for stable releases are usually filed by official developers for the project, and are called « pull-up requests » [15]. In most cases, this consists in either:

  • importing a fix from the development branch, or

  • updating a single package to a newer version, as released upstream and committed in the development branch of pkgsrc.

Pull-up requests are performed on the latest stable release only. In practice, this means that regular users of the project need to update to the newest stable releases as soon as they are released, in order to benefit from regular security updates. A criticism of the merits for this approach was already formulated in a separate paper[16], and will not be elaborated further here. It is however worth mentioning that Joyent, an industrial user of the pkgsrc project, offers Long Term Support (LTS) for a number of releases. The cloud-computing company keeps maintaining some stable releases for security issues a complete three years after the initial release[17].

The procedure within the official pkgsrc project is otherwise performed with the same infrastructure and rules as for the NetBSD project[18].

3 Hardening features

The pkgsrc project is already leveraging a number of features to help with the security of the software built. They are listed here in no particular order.

3.1 Package signatures

pkgsrc is one of the first Open Source software distributions to have introduced support for signing packages, as far back as in 2001[19]. While signing packages does not actually mitigate, even less solve, any potential vulnerability contained within a binary package, it does ensure the authenticity and integrity of the package being downloaded and installed. Without this support, it is impossible to know if a package may have been modified while in transit, on any hop from the remote server until reaching the system of the end-user.

When enforced, the verification of package signatures also prevents an attacker with access on a site hosting official packages from tampering with them, as long as the key material signing the packages is not compromised. This is particularly important in a context where many users use the protocols configured by default, both vulnerable to network interception: FTP and HTTP.

Although some issues actually prevented this feature to possibly be deployed until recently, a number of fixes and additions allowed Joyent to enable signature verification for its users, since their release of the 2014Q4 stable branch[20]. As noted there, this required modifying the original code to use libnetpgpverify instead of GnuPG. Signatures based on X.509 certificates are also supported, but often considered more complex to deploy and maintain than the variant based on GPG.

With the original code relying on GnuPG, a circular dependency was effectively in place, since the corresponding tool was only available within a package, security/gnupg. Jonathan Perkin solved this issue for the purpose of verifying signatures in September 2015[21]. This work is currently being completed by the author of this paper, and was submitted in its current state to the mailing-list[22].

As documented in the message accompanying these patches, a few challenges still have to be tackled before support for GPG-based signatures can effectively be migrated to libnetpgp and netpgp, and put in place without circular dependencies. This notably includes:

  • The need for a command-line wrapper interpreting options and arguments as expected by gpg to the way netpgp expects them instead (like the gpg2netpgp shell script linked above).

  • Issues with netpgpkeys when importing more than one key in a given keyring.

  • A security issue, where netpgp may report what looks like a detached signature as being successfully verified, whereas it will really be looking at content within the signature instead of the file to be verified.

The author of this paper hopes that this approach will help more software vendors to offer signed packages by default (including the NetBSD Foundation) while also easing the process for users to sign and verify their own packages as well.

3.2 Stack Smashing Protection (SSP)

This set of techniques aims at reducing the impact and exploitability of Buffer Overflow vulnerabilities [23]. It can imply a different layout for stack variables in memory, or the addition of « canaries ». These are values placed on the stack, which work as markers and whose integrity is checked when returning from routines. When the markers are found to have been modified, memory corruption is assumed and the program terminates itself (typically with the abort call). An illustration of a program flaw caught by SSP can be found in Figure 2, “Controlled crash with SSP”.

Figure 2. Controlled crash with SSP

#include <stdio.h>

int main(int argc, char * argv[])
{
        char buf[16];

        sprintf(buf, "Called as %s", argv[0]);
        printf("%s\n", buf);
        return 0;
}
$ gcc -o ssp ssp.c; ./ssp; /a/very/long/path/to/ssp; echo $?
Called as ./ssp
Called as /a/very/long/path/to/ssp
Segmentation fault (core dumped)
139
$ gcc -o ssp -fstack-protector ssp.c; ./ssp; /a/very/long/path/to/ssp; echo $?
Called as ./ssp
Called as /a/very/long/path/to/ssp
Abort trap (core dumped)
134

This option is typically implemented by the compiler, and enabled thanks to an additional command-line argument. A list of compilers and corresponding options is included in Table 1, “Compiler support for SSP”.

Table 1. Compiler support for SSP

CompilerOption
Clang/LLVM-fsanitize=address, -fsanitize=bounds, or with SafeCode
GCC-fstack-protector (protects only some vulnerable functions), -fstack-protector-strong (balanced for security and performance), or -fstack-protector-all (protects all functions)
IBM Compiler-qstackprotect
Microsoft Visual Studio/GS (default since 2005)

In pkgsrc, support for SSP can already be enabled with the PKGSRC_USE_SSP option in mk.conf. However, it requires the underlying platform, with the compiler in particular, to support this feature - and to be implemented in pkgsrc as such. At the moment, this is only the case for NetBSD, when building with GCC (on all architectures except alpha, hppa, ia64, and mips. Support can be extended to more platforms, beginning with those for which the relevant options were gathered here; this still requires testing before integration.

As can be observed with GCC or Clang/LLVM, different variants of SSP may be available from the compiler. This causes an issue with the configuration of SSP in pkgsrc, as the possible values for a variable choosing the flavour to use will not be meaningful on all platform combinations. A possible suggestion is to re-use PKGSRC_USE_SSP to choose among different flavours (such as strong or all) and then default back to the flavour used for yes when the value set is not understood and not set to no.

Furthermore, some packages may fail to build when enabling SSP. Except for bugs in the compiler itself, this is because SSP may impose additional restrictions while compiling C code. They can be related to the stack layout, like with dynamic allocation of memory on the stack using calloc. In this case, a patch can be written to use malloc instead. More complex patches may have to be applied in particular cases, as can be observed in audio/cdparanoia/patches/patch-cg.

Some packages may fail to run when enabling SSP, or crash where they would not be crashing without enabling SSP. This is usually because of existing bugs in the program, which are triggered when using SSP. Memory corruption bugs in particular are very likely to trigger crashes, instead of being silently ignored. SSP is therefore very useful in exposing these bugs.

In some particular contexts, the user may value availability more than safety or correctness. However, generally bugs are better exposed and fixed accordingly. With SSP, debugging can be performed as usual to identify the source of crashes: core dumps are currently generated by default.

It is possible for users to tell if at least part of the resulting binary was correctly built with SSP support enabled. This can be performed with the nm command, as executable binaries will pull in additional symbols for this purpose. Their names may vary according to the platform and compiler used though, as found in Figure 3, “Listing symbols for SSP (-fstack-protector) with GCC on NetBSD 7 (amd64)”.

Figure 3. Listing symbols for SSP (-fstack-protector) with GCC on NetBSD 7 (amd64)

00600c50 d _DYNAMIC
00600e30 d _GLOBAL_OFFSET_TABLE_
         w _Jv_RegisterClasses
00600c30 D __CTOR_LIST_END__
00400898 T ___start
00600ef4 B __bss_start
         w __deregister_frame_info@@GCC_3.0
00600ea8 D __dso_handle
00600ea0 D __progname
00600f40 B __ps_strings
         w __register_frame_info@@GCC_3.0
         U __stack_chk_fail
00600f00 B __stack_chk_guard
00400880 T __start
         U __syscall
00600ef4 D _edata
00600f68 B _end
         U _exit
00400ad0 T _fini
004007b0 T _init
         U _libc_init
00400880 T _start
         U atexit
00600f60 B environ
         U exit
00400a70 T main
         U puts
         U sprintf

Major projects outside of pkgsrc have already chosen to enforce the use of SSP by default: in Fedora and in Ubuntu Linux[24] since 2006, in OpenBSD since 2003 [25] and in DragonFlyBSD since 2013 or earlier[26]. While performance can be affected noticeably when forcing the use of SSP on every function call, Han Shen from Google submitted a more balanced approach, after heavy testing on their software and devices[27]. It is therefore suggested to adopt this approach the default in pkgsrc as well.

3.3 Fortify

Another technique consists in automatically adding boundary checks where possible. Called "fortify" in GCC, this technique is implemented in the compiler. It requires system headers conforming with this feature to really work, which usually involves support from the system's C library as well. This is the case on FreeBSD, Linux (glibc), NetBSD, OpenBSD, and possibly more platforms. An illustration of a program flaw caught by fortify can be found in Figure 4, “Controlled crash with fortify”.

Figure 4. Controlled crash with fortify

#include <stdio.h>

int main(int argc, char * argv[])
{
        char buf[32];

        sprintf(buf, "Called as %s", argv[0]);
        printf("%s\n", buf);
        return 0;
}
$ gcc -o fortify fortify.c; ./fortify; /a/very/long/path/to/fortify; echo $?
Called as ./fortify
Called as /a/very/long/path/to/fortify
0
$ gcc -o fortify -D_FORTIFY_SOURCE=2 -O2 fortify.c; ./fortify; /a/very/long/path/to/fortify
Called as ./fortify
Abort trap (core dumped)
134

With GCC, this feature is enabled by setting the pre-processing variable _FORTIFY_SOURCE to a value of 1 or 2, where 2 offers the strongest protection. This has to be done together with an optimization level of 1 or more. In practice, function calls to unsafe functions of the C library (like gets or sprintf) are replaced with alternative versions[28]. To possibly gain any benefit from these versions, they are enriched with calls built-in to the compiler, returning information about the parameters of the function. In some cases (e.g. static buffers) the compiler will readily know the size of a memory area about to be accessed, and place an additional boundary check. As an example, a call to the sprintf string formatter will be replaced with a call equivalent to snprintf, automatically setting the size parameter whenever compile-time information about its possibly lowest value might be.

Like SSP, support for this feature can already be leveraged in pkgsrc. This is enabled with the PKGSRC_USE_FORTIFY option. Some issues are already known, as they are equivalent to those from SSP support. They include:

  • Different compilers may have incompatible semantics, which will likely be reflected in the possible values for the option. This gets more complicated with the need for an optimization flag with GCC, which normally belongs (and may conflict with) existing values set in CFLAGS.

  • There is a minor performance impact, although not measurable in regular conditions[29]

  • Build failures when enabling fortify.

  • Unexpected behaviour or crashes when running programs.

Again like SSP, this feature adds memory safety checks in situations where an existing bug may have been silently ignored. Again, consequently, the program compiled may behave differently. Depending on the context, the user may prefer to keep running an unsafe version of the binary executable generated, but here again fortify is an efficient way to trigger and identify existing bugs.

Unfortunately, some programs may fail to compile when built with fortify. This can be the case when C programmers use macros with the same name as the functions being protected by fortify[30] (memcpy, mempcpy, memmove, memset, strcpy, stpcpy, strncpy, strcat, strncat, sprintf, vsprintf, snprintf, vsnprintf, gets).

Fortify is not a perfect solution for security, but it uncovers issues and mitigates a number of attacks at a negligible cost. Other software vendors have adopted it and enabled it by default already, such as Ubuntu Linux or Android. It should be considered as the default for pkgsrc as well.

3.4 Position-Independent Executables (PIE)

PIE binary executables are programs compiled in a particular way, allowing the processor instructions generated to perform regardless of the address at which the code is initially placed in memory [31]. While there might be a slight performance impact while running such programs [32], this feature can be used together with Address Space Layout Randomization (like implemented by PaX ASLR[33]) to load programs (and library code) at different, unpredictable positions in memory.

This effectively makes an entire range of computers attacks much more difficult to exploit, requiring an additional memory disclosure vulnerability in order to prepare code injected into a process at run-time accordingly. While the implementation of ASLR belongs to the underlying Operating System, pkgsrc can be configured to always try to build PIE binary executables, through the PKGSRC_MKPIE option. This involves compilation flags, and it is currently supported on NetBSD and GCC only in pkgsrc.

In practice, building code as PIE requires guessing if the current file is being compiled or linked as part of an executable program, as opposed to shared libraries for instance. Since it involves both the compilation and linking phases, it can only work for packages respecting the CFLAGS and LDFLAGS provided by pkgsrc. Unfortunately, many packages still fail to build in this scenario.

A compromise had to be made while implementing this feature in pkgsrc: the compilation parameter -fPIC is used instead of the more adequate -fPIE while creating executable programs. This is because it is effectively impossible for the pkgsrc wrappers to guess the purpose of a file being compiled: this is up to the original software and its build system in particular.

Another difficulty was encountered while linking programs: this has to be performed with the -pie parameter, but only when creating executables. This means that this parameter has to be added by the wrapper, as opposed to being part of the LDFLAGS parameter. While this was implemented in the wrapper code in pkgsrc's mk/wrapper directory, this is not reflected in the pkgtools/cwrappers package as of the time writing this paper. With cwrappers now being the default wrapper system in place in pkgsrc, this means that PIE support is currently broken on its own in pkgsrc, and so requires setting USE_CWRAPPERS=no in mk.conf.

The EdgeBSD Project[34] offers a package for the purpose of verifying the proper enforcement of various security features in pkgsrc, with PIE and ASLR in particular[35]. It helps determine if the current settings are correctly reflected when building packages. It is illustrated in Figure 5, “Output of edgebsd/hardening without PIE” for the default settings and in Figure 6, “Output of edgebsd/hardening with PIE and ASLR, SSP, FORTIFY, and RELRO enabled”. These results were obtained while running EdgeBSD 7 (amd64), a clone of the netbsd-7 branch with additional hardening patches (currently only available from source).

Figure 5. Output of edgebsd/hardening without PIE

pkgsrc/edgebsd/hardening$ bmake install && ~/pkg/bin/hardening && file ~/pkg/bin/hardening
[...]
===> Building for edgebsd-hardening-3
cc    -O2  -fPIC -o lib.o -c lib.c
ar -rc libHardening.a lib.o
ranlib libHardening.a
cc -shared -o libHardening.so.0.0 -Wl,-soname,libHardening.so.0 lib.o  -Wl,-R/home/khorben/pkg/lib
ln -f -s -- libHardening.so.0.0 libHardening.so.0
ln -f -s -- libHardening.so.0.0 libHardening.so
cc    -O2  -o main.o -c main.c
cc -o hardening main.o  -Wl,-R/home/khorben/pkg/lib -L. -lHardening
=> Unwrapping files-to-be-installed.
===> Installing for edgebsd-hardening-3
[...]
===> Installing binary package of edgebsd-hardening-3
[!] Hi! I am a library.
[!] Let's see if I am strong enough...
[+] built with -fPIC
[!] Bye! I am not a library anymore.
[!] Hi! I am an executable.
[-] NOT built with -fPIE or even -fPIC, no complete ASLR
[-] not built with _FORTIFY_SOURCE at all :(
[+] mmap() failed W|X, good
[-] mmap() gave two identical addresses :(
/home/khorben/pkg/bin/hardening: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked (uses shared libs), for NetBSD 7.1, not stripped

Figure 6. Output of edgebsd/hardening with PIE and ASLR, SSP, FORTIFY, and RELRO enabled

pkgsrc/edgebsd/hardening$ bmake install && hardening && file ~/pkg/bin/hardening
[...]
===> Building for edgebsd-hardening-3
cc    -O2 -fPIC -D_FORTIFY_SOURCE=2 -fPIC -o lib.o -c lib.c
ar -rc libHardening.a lib.o
ranlib libHardening.a
cc -shared -o libHardening.so.0.0 -Wl,-soname,libHardening.so.0 lib.o  -Wl,-z,relro -Wl,-z,now -Wl,-R/home/khorben/pkg/lib
ln -f -s -- libHardening.so.0.0 libHardening.so.0
ln -f -s -- libHardening.so.0.0 libHardening.so
cc    -O2 -fPIC -D_FORTIFY_SOURCE=2 -o main.o -c main.c
cc -o hardening main.o  -Wl,-z,relro -Wl,-z,now -Wl,-R/home/khorben/pkg/lib -L. -lHardening
=> Unwrapping files-to-be-installed.
===> Installing for edgebsd-hardening-3
[...]
===> Installing binary package of edgebsd-hardening-3
[!] Hi! I am a library.
[!] Let's see if I am strong enough...
[+] built with -fPIC
[!] Bye! I am not a library anymore.
[!] Hi! I am an executable.
[+] built with -fPIC, good enough for full ASLR
[+] built with _FORTIFY_SOURCE 2, all good
[+] mmap() failed W|X, good
[-] mmap() gave two identical addresses :(
/home/khorben/pkg/bin/hardening: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked (uses shared libs), for NetBSD 7.1, not stripped

The default compilation settings of the package are limited to the minimum allowing a successful build on most platforms, which is why the library linked was always built with -fPIC. As illustrated, it is possible for programs to detect the use of fortify while compiling, as well as the generation of PIC or PIE code. While the kernel used was patched to enforce W^X (code execution only in read-only memory zones), the implementation of ASLR is still not perfect, as two successive calls of mmap returned the same address after unmapping the zone.

Another important aspect illustrated here is the output of the file command on the resulting executable program. It is possible to tell if the program was effectively built as a PIE binary, since its ELF header will be that of a shared object, instead of an executable.

3.5 RELRO and BIND_NOW

RELRO protects ELF executable programs against tampering at run-time. Additional sections of the program loaded in memory are made read-only once relocations are performed. This reduces the attack surface for possible exploits against the program. This approach can be combined with the resolution of all dynamic symbols at start-up (instead of on-demand, also known as "immediate binding") so that the GOT section in particular can be made read-only as well.

Enforcing RELRO or BIND_NOW through pkgsrc involves the linking phase while compiling programs. Like PIE, it is performed through the LDFLAGS parameter, and it only works for packages respecting this parameter. The corresponding parameters for GCC are -Wl,-z,relro and -Wl,-z,now for RELRO and BIND_NOW, respectively. Packages will however not fail to build if this option is not caught up by the build system from the package. This also means that failures to reflect this requirement in packages will be silently ignored.

It is possible for users to check if a binary or library was successfully built with RELRO. An additional entry in the program header table, RELRO, will be found there. This can be determined with the objdump command, as illustrated in Figure 7, “Output of objdump without RELRO” and Figure 8, “Output of objdump with RELRO”.

Figure 7. Output of objdump without RELRO

$ objdump -x hardening
[...]
Program Header:
    PHDR off    0x00000040 vaddr 0x00000040 paddr 0x00000040 align 2**3
         filesz 0x00000188 memsz 0x00000188 flags r-x
  INTERP off    0x000001c8 vaddr 0x000001c8 paddr 0x000001c8 align 2**0
         filesz 0x00000017 memsz 0x00000017 flags r--
    LOAD off    0x00000000 vaddr 0x00000000 paddr 0x00000000 align 2**21
         filesz 0x00001090 memsz 0x00001090 flags r-x
    LOAD off    0x00001090 vaddr 0x00201090 paddr 0x00201090 align 2**21
         filesz 0x0000031c memsz 0x00000358 flags rw-
 DYNAMIC off    0x000010b8 vaddr 0x002010b8 paddr 0x002010b8 align 2**3
         filesz 0x000001c0 memsz 0x000001c0 flags rw-
    NOTE off    0x000001e0 vaddr 0x000001e0 paddr 0x000001e0 align 2**2
         filesz 0x0000002c memsz 0x0000002c flags r--
EH_FRAME off    0x00000fb8 vaddr 0x00000fb8 paddr 0x00000fb8 align 2**2
         filesz 0x00000024 memsz 0x00000024 flags r--

Figure 8. Output of objdump with RELRO

$ objdump -x hardening
Program Header:
    PHDR off    0x00000040 vaddr 0x00000040 paddr 0x00000040 align 2**3
         filesz 0x000001c0 memsz 0x000001c0 flags r-x
  INTERP off    0x00000200 vaddr 0x00000200 paddr 0x00000200 align 2**0
         filesz 0x00000017 memsz 0x00000017 flags r--
    LOAD off    0x00000000 vaddr 0x00000000 paddr 0x00000000 align 2**21
         filesz 0x000010d0 memsz 0x000010d0 flags r-x
    LOAD off    0x00001d18 vaddr 0x00201d18 paddr 0x00201d18 align 2**21
         filesz 0x0000033c memsz 0x00000370 flags rw-
 DYNAMIC off    0x00001d40 vaddr 0x00201d40 paddr 0x00201d40 align 2**3
         filesz 0x000001e0 memsz 0x000001e0 flags rw-
    NOTE off    0x00000218 vaddr 0x00000218 paddr 0x00000218 align 2**2
         filesz 0x0000002c memsz 0x0000002c flags r--
EH_FRAME off    0x00000ff8 vaddr 0x00000ff8 paddr 0x00000ff8 align 2**2
         filesz 0x00000024 memsz 0x00000024 flags r--
   RELRO off    0x00001d18 vaddr 0x00201d18 paddr 0x00201d18 align 2**0
         filesz 0x000002e8 memsz 0x000002e8 flags r--

The correct execution of programs is not expected to be affected by RELRO or BIND_NOW, except in the case of missing symbols: the program will not be allowed to start. However, a performance penalty is induced by this approach. While it is negligible for small programs, it can slow down large executables while loading, since they will have to lookup every symbol for every library used, immediately upon startup. While it is suggested to enable RELRO by default in pkgsrc as well, it would probably make sense to limit it to lazy binding (as opposed to immediate binding) at least until a work-around or solution is found in this case.

On the other hand, background programs from the Operating System (e.g. system daemons) would only be affected once, when starting. Most network servers are started this way and are among the most important programs to protect. This should also be taken into consideration when choosing whether and how to enable support for RELRO by default.

Some major software distributions have already decided to enable RELRO by default, like Ubuntu Linux. OpenBSD has even enabled RELRO for all dynamic executables and libraries, as well as for static PIE executables, by default since August 2016[36]. Unfortunately, in pkgsrc, like most of the hardening features detailed in this paper, support for RELRO is currently limited to GCC on NetBSD.

4 Future work

Obviously, the features detailed so far will have to be tested further, extended for more platforms, and enabled by default where relevant. Some possible directions for additional hardening features are introduced here.

4.1 Reproducible builds

The aim of reproducible builds is to provide a verifiable path from source code to binary. A community was formed around this principle, leading an effort to popularize this practice[37]. The definition proposed by this community is as follows:[38]

 

A build is reproducible if given the same source code, build environment and build instructions, any party can recreate bit-by-bit identical copies of all specified artifacts.

The relevant attributes of the build environment, the build instructions and the source code as well as the expected reproducible artifacts are defined by the authors or distributors. The artifacts of a build are the parts of the build results that are the desired primary output.

 
 --reproducible-builds.org

The FreeBSD ports have already adopted this practice by default, with 64,25% of all packages of all packages readily built this way[39]. Given the proximity of this project to pkgsrc, it should be possible to adapt this approach to pkgsrc as well. With the base system for NetBSD already built reproducibly for official releases, it should also be possible to extend it to official packages from pkgsrc as well.

4.2 Code Flow Integrity

Control-flow integrity (CFI) is a general term for computer security techniques which prevent a wide variety of malware attacks from redirecting the flow of execution of a program[40]. An implementation is notably available in Clang[41]. Although bypassing techniques also exist here, it is an additional and promising candidate to experiment with. Seemingly involving the use of specific parameters while compiling software, the pkgsrc project would again be a great test-bed for this feature.



[1] pkgsrc, portable package build system, https://pkgsrc.org/

[2] The NetBSD Project, https://www.NetBSD.org/

[4] Joyent Packages Documentation, https://pkgsrc.joyent.com/

[12] daily.conf(5) - NetBSD Manual Pages, http://man.NetBSD.org/daily.conf.5

[14] pkg_install.conf(5) - NetBSD Manual Pages, http://man.NetBSD.org/pkg_install.conf.5

[15] Pullup ticket tracking summary for NetBSD pkgsrc, http://releng.NetBSD.org/index-pkgsrc.html

[17] Joyent | pkgsrc-2014Q4: LTS, signed packages, and more, https://www.joyent.com/blog/pkgsrc-2014q4-lts-signed-packages-and-more

[19] tech-pkg: [[email protected]: CVS commit: basesrc/usr.sbin/pkg_install/add], https://mail-index.NetBSD.org/tech-pkg/2001/09/25/0005.html

[20] Joyent | pkgsrc-2014Q4: LTS, signed packages, and more, https://www.joyent.com/blog/pkgsrc-2014q4-lts-signed-packages-and-more

[22] Handling GPG signatures for pkgsrc with netpgp, http://mail-index.NetBSD.org/tech-pkg/2017/02/02/msg017766.html

[27] [PATCH] Add a new option "-fstack-protector-strong" (patch / doc inside), https://gcc.gnu.org/ml/gcc-patches/2012-06/msg00974.html

[29] GCC 4.1 Release Series ­ Changes, New Features, and Fixes, https://gcc.gnu.org/gcc-4.1/changes.html

[30] Enhance application security with FORTIFY_SOURCE, https://access.redhat.com/blogs/766093/posts/1976213

[32] Position Independent Executable (PIE) Performance, https://access.redhat.com/blogs/766093/posts/1975803

[34] The EdgeBSD Project, https://www.edgebsd.org/

[37] reproducible-builds.org, https://reproducible-builds.org/