go-audit: Why Slack Rewrote Linux’s Audit Daemon in Go
Hook
Linux’s auditd has been blocking kernel operations and dropping events under load for years. Slack got tired of it and built a replacement that treats audit logs like streaming data instead of system call interruptions.
Context
If you’ve run auditd in production, you’ve likely encountered challenges. The traditional Linux audit daemon connects to the kernel’s audit subsystem to record security-relevant events—file access, process execution, and network connections your compliance team requires. But auditd was designed in an era when blocking I/O was standard practice.
Modern infrastructure teams need audit data flowing into systems like Elasticsearch or Graylog in real-time. The traditional approach meant running auditd, then audisp plugins to transform logs, then forwarders to ship them. Each hop introduced latency and potential failure points. Under high system activity, auditd could block or drop events, creating gaps in audit trails. Slack’s team faced these issues and decided the solution wasn’t another plugin—it was building a new daemon in Go that could handle concurrent I/O more effectively.
Technical Insight
go-audit takes a different approach by connecting directly to the Linux kernel’s netlink socket interface. Netlink is the kernel’s mechanism for passing structured messages between kernel and user space, and the audit subsystem exposes events through it. The README indicates go-audit is written in Go to be ‘type safe and performant’ with the goal to ‘never ever ever ever block if we can avoid it.’
The core advantage is JSON output. While traditional auditd produces text logs that need parsing, go-audit outputs JSON natively, which the README lists as one of its primary goals. This eliminates downstream parsing infrastructure—your logging pipeline receives structured data from the start.
The tool supports pluggable outputs, as documented in the README: ‘Can write to syslog, local file, Graylog2 or stdout. Additional outputs are easily written.’ This means you can simultaneously log locally and forward remotely without managing separate audisp plugins.
A critical configuration option is socket buffer management. If go-audit can’t consume messages as fast as the kernel produces them, you’ll see ‘no buffer space available’ errors. The README’s FAQ addresses this directly, recommending you increase the socket_buffer.receive configuration value:
socket_buffer:
receive: <some number bigger than (the current value * 2)>The README advises that reducing audit velocity is preferable, but if that’s not possible, increasing the receive buffer may help. The tool appears designed to handle high-throughput audit streams more gracefully than traditional auditd, though the specific implementation details of how Go’s concurrency features are leveraged aren’t documented in the README.
Gotcha
go-audit inherits a fundamental limitation from the Linux kernel itself: it often can’t resolve filenames from inodes. The README’s FAQ explicitly addresses this: ‘The kernel doesn’t always know the filename for file access. Figuring out the filename from an inode is expensive and error prone.’ You’ll see audit records with inode information but no filename, requiring manual investigation. The README suggests mapping back using debugfs:
sudo debugfs -R "ncheck <inode to map>" /dev/<your block device here>The ‘no buffer space available’ error can still occur under extreme audit load, as the README acknowledges. While go-audit’s design appears to help compared to auditd, the FAQ states: ‘This is because go-audit is not receiving data as quickly as your system is generating it.’ Even with increased socket buffers, physics remains a constraint. The recommended solution is ‘try and reduce the amount of data go-audit has to handle.’
go-audit is Linux-specific and tightly coupled to the netlink audit API, as the README notes it ‘Connects to the linux kernel via netlink.’ If you need audit logging on other operating systems, this tool won’t work. The README also doesn’t discuss integration with existing auditd tooling or SELinux utilities, so compatibility with tools expecting traditional audit.log format may be a concern.
Verdict
Use go-audit if you’re running Linux in production and need audit data in JSON format for centralized logging systems. The native JSON output eliminates log parsing infrastructure, and the pluggable pipeline supports simultaneous outputs (local files, syslog, Graylog) without managing multiple plugins. Consider go-audit if you’ve experienced issues with traditional auditd under load, particularly the blocking and event dropping that the tool was designed to address. The fact it’s written in Go means single-binary deployment, which can simplify operations. Skip go-audit if your audit requirements are minimal and traditional auditd meets your needs—the advantages primarily materialize under higher audit velocity. Also consider carefully if you’re heavily invested in auditd-specific tooling or need cross-platform support beyond Linux. The inode-to-filename resolution limitation affects both auditd and go-audit, so if complete filename information is critical and you can’t tolerate any inode-only records, additional tooling will be needed regardless of which daemon you choose.
