CHANGELOG.md

osquery Changelog

5.0.1

Git Commits

Representing commits from 21 contributors! Thank you all.

osquery 5.0 is a tremendously exciting release!

• We now install into /opt/osquery on macOS and Linux for better portability.
• Our default and recommended installation for macOS uses an application bundle to support entitlement-based features.
• We now use Endpoint Security APIs for various event-based tables on macOS (more to come in the future!)
• We now use an osquery-organization macOS code signing certificate.

There are several breaking changes:

• Installation paths have changes from /usr/local to /opt/osquery on macOS and Linux (symlinks to executables are provided).
• macOS codesigning is now down through the Osquery Foundation account
• If you manage macOS full disk permission through a profile, you will need to update it. See docs
• We removed the deprecated blacklist key from the configuration (#7153)
• Search semantics on the augeas table have changed to be more performant, but do break the existing query API.

Table Changes

• Add secureboot table for Linux and Windows (#7202)
• Add tpm_info for Windows (#7107)
• Fix osquery_info build_platform column value on Linux (#7254)
• Support pid_with_namespace in more tables (#7132)
• Update augeas table to use native pattern matching (BREAKING) (#6982)
• Update chrome_extensions to include Edge & EdgeBeta (#7170)
• Update disk_encryption table to support QueryContext (#7209)
• Update last to include utmp type name column (#7201)
• Update sudoers table to support newer include syntax (#7185)
• Update user_ssh_keys to detect encryption of ed25519 keys (#7168)

Under the Hood Improvements

• Add ruby namespace to the thrift definition (#7191)
• Always initialize variable change in PerformanceChange (#7176)
• Remove deprecated blacklist key (#7153)
• Use total_size within watchdog on Windows (#7157)
• Support AF_PACKET sockets reporting on Linux (#7282)
• socket_events improvements in Linux audit system (#7269)

Bug Fixes

• Add case sensitive pragma to the pragma/actions authorizer allow list (#7267)
• Add feature to skip denylist for event-based queries (#7158)
• Change logger_mode flag to be correctly interpreted as an octal (#7273)
• Do not let osquery create multiple copies of the extension running at once (#7178)
• Fix Linux audit rule removal upon osquery exit (#7221)
• Fix broadcasting empty logs to logger plugins (#7183)
• Fix issues applying ACLs during chocolatey deployment (#7166)
• Fix memory issue in Windows fileops (#7179)
• Fix process_open_sockets type error on darwin (#6546)
• Make sure that the file action MOVED_TO is tracked with yara events. (#7203)
• Prevent osquery from killing itself when the --force flag is used (#7295)
• Prevent race condition between shutdown and worker or extension launch (#7204)

Documentation

• Add a security assurance case (#7048)
• Bring the YARA wiki page up to date (#7172)
• Spelling fixes (#7211, #7186)
• Update uptime table description (#7270)
• Update osquery installed artifacts paths in the documentation (#7286)

Build

• Add TimeoutStopSec to systemd service files (#7190)
• Correct macOS installed app bundle path in osqueryctl and doc (#7289)
• Create an macOS app bundle (#7263)
• Fix choco packaging not failing when an error occurs during install or upgrade (#7182)
• Fix path in macOS launchd plist (#7288)
• Pin the packaging repo within GitHub workflows (#7208, #7255, #7279)
• Update Windows deployment icon to png (#7163)
• Update install paths, and remove deprecated Facebook naming (#7210)
• Update macOS build to include app bundle related files (#7184)
• Update osquery installed artifacts default paths in code (#7285)
• Update the installation path on Linux (#7271)
• libs: Add options to AWS Optionally enable debug option and restrict content-type header size for PUT req (#7216)
• libs: Enable and compile the YARA macho module on macOS (#7174)
• libs: Update OpenSSL to version 1.1.1l (#7293)
• libs: Update Strawberry Perl to 5.32.1.1, use HTTPS downloads (#7199)
• libs: Update ebpfpub (#7173, #7219)

4.9.0

Git Commits

Representing commits from 16 contributors! Thank you all.

New Features

• Add filesystem logrotate feature (#7015)
• Add Non-Functional EndpointSecurity based process events to macOS (Requires updated codesigning due in 5.0) (#7046)

Table Changes

• Add mdm_managed column to system_extensions on macOS (#6915)
• Add prefetch table on Windows (#7076)
• Add support for IMDSv2 to AWS tables (#7084)
• Enable container stats on docker containers that don't have traditional networks (#7145)
• Update homebrew_packages to include new prefix, and allow specifying alternate prefixes (#7117)
• Update ntfs_acl_permissions to list all ACE entries (using GetAce()) (#7114)
• Update processes table to display additional Windows attributes (secured, protected, virtual, elevated) (#7121)
• Update how package_install_history identifies the packageIdentifiers key (#7099)
• Update how identifier is calculated in chrome_extensions (#7124)

Under the Hood improvements

• Improve speed of osquery shutdown procedure (#7077)
• Improve shutdown speed during initialization (#7106)
• Update website generators (#7136)
• CLI flag to allow osquery to keep retrying enrollment (instead of exiting) (#7125)
• rocksdb: Do not fsync WAL writes (#7094)
• Move CPack packaging to a dedicated repository (#7059)
• Restore thrift socket 5min timeout (#7072)
• Consolidate syscalls to a single audit rule (#7063)

Bug Fixes

• Add current WMI location for Dell BIOS info (#7103)
• Correct RocksDB error code and subcode printing on open failure (#7069)
• Fix pipe_channel not reading all data in a message (#7139)
• Fix crash and deadlocks in recursive logging (#7127)
• Fix custom curl_certificate timeouts (#7151)
• Fix extensions crash on shutdown (#7075)
• Handle updated paths on various macOS tables -- xprotect_entries, xprotect_meta, launchd (#7138, #7154)
• Trigger event cleanup checks every 256 events (#7143)
• Update generating an extension uuid to be thread safe (#7135)
• Watchdog should wait for the worker to shutdown (#7116)

Documentation

• Update process auditing requirements documentation (#7102)
• Update website docs indicating windows support for YARA tables (#7130)
• Add 4.9.0 CHANGELOG (#7152)

Build

• Add Apple provisioning profile for distribution (#7119)
• Add more tests for events expiration (#7071)
• CI: Regenerate sccache cache when compiler version changes (#7081)
• Fix flaky test test_daemon_sigint by waiting for pidfile (#7095)
• Fix icon in Windows packaging (#7148)
• Minor cleanup of unused variables (#7128)
• Print extension SDK minimum version required when failing to load (#7074)
• Remove POSIX-only -fexceptions flag on Windows (#7126)
• Remove duplicated osquery_utils_aws_tests-test (#7078)
• Remove flaky test decorators for python tests (#7070)
• Update SQLite to version 3.35.5 (#7090)
• Update librdkafka to version 1.7.0 (#7134)
• Update libyara to version 4.1.1 (#7133)

4.8.0

Git Commits

Representing commits from 14 contributors! Thank you all.

This version fixes a regression introduced in 4.7.0 related to events expiration optimization. Please read (#7055) for more information.

This release upgrades openssl, as is general good practice. Osquery is not known to be effected by any security issues in OpenSSL.

New Features

• shell: Add .connect meta command (#6944)

Table Changes

• Add seccomp_events table for Linux (#7006)
• Add shortcut_files table for Windows (#6994)

Under the Hood improvements

• Removing Keyboard Event Taps from osx-attacks pack (#7023)
• Refactor watcher out of singleton pattern (#7042)
• Small events subscriber refactor to increase test coverage (#7050)
• Setting non-required deb_packages fields as optional in test (#7001)

Bug Fixes

• Handle events optimization edge cases (#7060)
• Fix optimization for multiple queries using the same subscriber (#7055)
• Use epoch and counter for events-based queries (#7051)
• Guard node key to prevent duplicate enrollments (#7052)
• Change windows calculation for physical_memory (#7028)
• Free using WTSFreeMemoryEx for WTSEnumerateSessionsExW (#7039)
• Release variable in Windows data conversation (#7024)
• Change chrome_extensions warnings to verbose (#7032)
• Add transactions to the SQLite authorizer PRAGMAs (#7029)
• Change Windows messages to verbose (#7027)
• Fix scheduler to print the correct number of elapsed seconds (#7016)

Documentation

• Fix tls_enroll_max_attempts flag name in the documentation (#7049)
• Improve docs on FIM, mention NTFS and Audit, etc. (#7036)
• config: Add docs for the events top-level-key (#7040)
• Add funding link on GitHub generated page (#7043)
• Correct the example in the windows_events table spec (#7035)
• Correct docs about OpenSSL and TLS behavior (#7033)
• Update docs to describe how to build for aarch64/arm64 (#6285) (#6970)
• Add a note on enabling Windows to build with CMake's long paths (#7010)
• Add 4.8.0 CHANGELOG (#7057)

Build

• Add an option to enable incremental linking on Windows (#7044)
• Remove Buck leftovers that supported building with old versions of OpenSSL (#7034)
• Add build_aarch64 workflow for push (#7014)
• Move CI to using docker from osquery (#7012)
• Update dockerfile to multiplatform (#7011)
• Run GH Actions workflows on all tags (#7004)
• Disable BPF events tests if OSQUERY_BUILD_BPF is false (#7002)
• libs: Update OpenSSL to version 1.1.1k (#7026)

4.7.0

Git Commits

Commits from 21 contributors! Thank you all!

New Features

• Add concat and concat_ws sql functions (#6927)
• Update the scheduler to log the query name at info level (#6934)
• Add support for SQLite RPM databases (#6939)

Table Changes

• Add computer column to Windows Eventlogs (#6952)
• Add docker_image_history table (#6884)
• Add filevault_status column to disk_encryption table (#6823)
• Add location_services table on macOS (#6826)
• Add shellbags table (#6949)
• Add system_extensions table on macOS (#6863)
• Add systemd_units table (#6593)
• Add ycloud_instance_metadata table (#6961)
• Fix loading of YARA rules on Windows (#6893)
• Fix macOS OpenDirectory attribute mismatch (#6816)
• Update augeas table not to autoload system lenses (#6980)
• Update chrome_extensions table -- more browser support and tests (#6780)
• Update office_mru table to correct platforms (#6827)
• Update aws table to include macOS (#6817)

Under the Hood improvements

• Remove Azure Pipelines (#6953)
• Disable deprecated TLS versions 1.0, 1.1 (#6910)
• Use librpm bdb_ro backend and remove bdb (#6931)
• bpf: Improve execve/execveat tracing, add AArch64 build support (#6802)
• Use a distinct carver request_id and add this to the schema (#6959)
• Initialize TLSLogForwarder before enrollment check (#6958)
• Put noisy thrift logs behind a flag (#6951)
• Fix bug in windows thrift, causing named pipe closing (#6937)
• Remove unused/experimental ebpf code (#6879)
• Remove unused ev2 code (#6878)
• Refactor the eventing framework to reduce disk IO and improve performance(#6610)

Bug Fixes

• Add journal_mode to the sqlite authorizer PRAGMAs (#6999)
• Add table_info to the sqlite authorizer PRAGMAs (#6814)
• Always use BIGINT macro for long long data (#6986)
• Copy JSON objects to avoid MemoryPool buildup (#6957)
• Do not call unconfigured subscribers errors (#6847)
• Do not ignore mountpoints that have the same mount path (#6871)
• Do not start scheduler when shutting down (#6960)
• Don't mark scope and key columns as index in selinux_settings table (#6872)
• Fix augeas table output bug for non-path entries (#6981)
• Fix pids column in docker_container_stats table (#6965)
• Fix additional relative path check in Yara for Windows (#6894)
• Fix config validation oom with duplicated keys (#6876)
• Fix data type macro used for 64-bit timestamp variables (#6897)
• Fix error in process_open_files inode need stoul, not stoi (#6983)
• Fix leaks when a query fails from the shell (#6849)
• Fix mem leak regression with Windows sids API (#6984)
• Make Group ID columns consistent across Windows tables (#6987)
• When iterating /proc, use individual try/catch so catch partial failures (#6933)
• augeas: Clear aug pointer on error (#6973)

Documentation

• Add 4.6.0 CHANGELOG (#6809)
• Add 4.7.0 CHANGELOG (#6985)
• Add docs for TLS enroll max attempts (#6888)
• Change reference about Azure Pipelines to GitHub Actions (#6988)
• Clarify FIM exclude category documentation (#6966)
• Document retrieval of available tables/columns via SQL (#6812)
• Fix Github Actions status badge in the README (#6908)
• Fix all broken or redirected URLs and references (#6835)
• Fix broken URL in docs (#6882)
• Fix incorrect Slack URLs (#6844)
• Fix packs discovery queries documentation (#6946)
• Fix reference to a Powershell script on Windows (#6936)
• Fix typos in source code (#6901)
• Improve explanations of event control flags (#6954)
• Spellcheck and Markdown edits (#6899)
• Update README to include release process comment (#6877)
• Update documentation about denylist schedule key (#6922)
• Update macOS OpenBSM configuration (#6916)
• Update the Linux install steps and package listing (#6956)
• Update the info about osquery's TLS version support (#6963)

Build

• CI: Add a RelWithDebInfo Linux job to generate packages (#6838)
• CI: Add support for GitHub Actions (#6885)
• CI: Add unit tests for RPM DB querying (#6919)
• CI: Fix ExtendedAttributesTableTests failing due to an unexpected attribute (#6942)
• CI: Fix StartupItemTest failing due to unexpected values (#6940)
• CI: Fix SystemControlsTest adding sunrpc as an expected subsystem (#6932)
• CI: Fix XattrTests failing due to unexpected attribute name (#6941)
• CI: Fix an incorrect check in StartupItems test (#6950)
• CI: Fix wifi_tests on macOS 10.15 and above (#6724)
• CI: Move cppcheck step after the tests (#6845)
• CI: Permit running formatting earlier in the CI (#6836)
• CI: Remove incorrect 2to3 symlink breaking Python brew upgrade (#6819)
• CI: Remove unused empty test file (#6918)
• CI: Remove unused tests for Rocksdb and Inmemory db plugins (#6900)
• CI: Update XCode to 12.3 and Update min macOS version to 10.12 (#6896, #6913)
• CI: Update macOS agent to 10.15 Catalina (#6680)
• CMake: Add -pthread compile option on posix platforms (#6909)
• CMake: Add Valgrind support (#6834)
• CMake: Add an option to disable building AWS tables and library (#6831)
• CMake: Add an option to disable building libdpkg tables and library (#6848)
• CMake: Detect missing headers during include namespace generation (#6855)
• CMake: Do not attempt to dllimport Thrift symbols (#6856)
• CMake: Do not compile Windows libraries with debug symbols (#6833)
• CMake: Explicitly set the MSVC runtime library (#6818)
• CMake: Fix amalgamated tables generation on change (#6832)
• CMake: Fix platformtablecontaineripc include namespace generation (#6853)
• CMake: Further fix amalgamation file gen on change (#6854)
• CMake: Refactor and rename fuzzers build flag (#6829)
• CMake: Significantly speed up configuration phase (#6914)
• CMake: Use make jobserver for OpenSSL on Linux and macOS (#6821)
• CPack: Remove extraneous lenses directory for augues on macOS (#6998)
• Change libdpkg submodule url to our own GitHub mirror (#6903)
• Disable incremental linking to reduce build size on Windows (#6898)
• GitHub Actions: Fix .deb artifacts, add scheduled builds (#6920)
• Remove hash and yara table from fuzz harnesses (#6972)
• libraries: Reduce the compilation units from libarchive (#6886)
• libraries: Remove the last usage of sqlite3 from sleuthkit (#6858)
• libraries: Rename yara str functions to avoid symbol collisions (#6917)
• libraries: Update librpm to version 4.16.1.2 (#6850)
• libraries: Update openssl to version 1.1.1i (#6820)
• libraries: Update thrift to version 0.13.0 (#6822)

Hardening

• Update CODEOWNERS to reflect existing teams (#6955, #6975)
• Restrict access to Thrift server pipe on Windows (#6875)
• Fix a leak in libdpkg when querying the deb_packages table (#6892)
• Fix UB and dangerous casting in the pubsub framework (#6881)
• Fix heap-use-after-free in deregisterEventSubscriber (#6880)
• Thift patch to support security configuration (#6846)
• Improve config fuzzer dictionary creation script (#6860)
• Avoid running queries for views when fuzzing (#6859)
• Improve fuzzing speed and stack trace accuracy (#6851)

4.6.0

Git Commits

New Features

• Initial implementations for BPF-based socket and process events tables (#6571)
• Support EC2 tables on Windows (#6756)

Under the Hood improvements

• BPF: Add container support to fork/vfork/clone (#6721)
• BPF: Additional improvements on the initial implementation (#6717)
• BPF: Fix the tests (#6783)
• BPF: Fix wrong d_type compare in filesystem classes (#6774)
• BPF: Implement additional syscalls to track file descriptor usage (#6723)
• Remove unused LTCG flag (#6769)
• Support TLS client certificate chains (#6753)
• Refactor carver to use the Scheduler (#6671)
• Add configuration flag to disable file_events by default (#6663)
• libs: Build x86_64 configurations on Ubuntu 14.04 (#6687)
• libs: Port the RocksDB Win7 compatibility patch to the MSBuild generator (#6765)
• libs: Update BPF libraries to support LLVM 11 (#6775)
• libs: Update RocksDB to version 6.14.5 (#6759)
• libs: Update bzip2 to version 1.0.8 (#6786)
• libs: Update ebpfpub to latest version (#6757)
• libs: Update sqlite to version 3.34.0 (#6804)
• libs: update aws-sdk to 1.7.230 (#6749)
• Adding support for pretty-printing JSON results in osqueryi (#6695)

Table Changes

• Add Yandex Browser support for chrome_extensions (#6735)
• Add additional file stat flags to Darwin (bsd_flags) (#6699)
• Add extended_attributes table to Linux, add support for Linux capabilities (#6195)
• Add indexed column support to Windows users table (#6782)
• Enable AWS Instance profile as credential provider on Windows (#6754)
• Add systemd support for startup_items on Linux (#6562)

Bug Fixes

• Do not use memset on VirtualTable, a non-POD type (#6760)
• Fix deadlock when registering two extensions (#6745)
• Fix last_connected column in wifi_networks on Catalina (#6669)
• Fix missing negations, duplicate rows in iptables table (#6713)
• Fix shadow table to detect empty passwords (#6696)
• Free memory allocated by ConvertStringSidToSid (#6714)
• PackageIdentifiers are optional in InstallHistory.plist (#6767)
• Removing PUNYCODE flag from windows string conversions (#6730)
• Fix memory leak in the dbus classes (#6773)
• Change the kernel_modules size column type to BIGINT (#6712)

Documentation

• Add a README.md to source-based libraries (#6686)
• Fix spelling typos (#6705)
• Journald Audit Logs Masking Documentation (#6748)

Build

• CI: Provide built packages as Azure artifacts (#6772)
• CI: Python installation improvements on Windows (#6764)
• CI: Update brew scripts (#6794)
• CMake: Disable BPF support if the LLVM libs are not compatible (#6746)
• CMake: Use CPACK_RPM_PACKAGE_RELEASE (#6805)
• CMake: Add max version limit to 3.18.0 on Linux (#6801)
• Change urls for submodules gpg-error, libgcrypt, libcap (#6768)
• Reduce linkage requirements for tests (#6715)
• Remove a Buck leftover (#6799)
• Remove boost workaround introduced in #5591 for string_view (#6771)
• Tests: Fix tests on Catalina (#6704)
• Update cmake_minum_required to 3.17.5 and pin version in CI (#6770)
• build: Fix Windows build on newer MSVC (#6732)
• extensions: Always compile examples to prevent them from breaking (#6747)

Security Issues

• Add SQLite authorizer to mitgate CVE-2020-26273 / GHSA-4g56-2482-x7q8 (https://github.com/osquery/osquery/commit/c3f9a3dae22d43ed3b4f6a403cbf89da4cba7c3c)

Packs

• Updated unwanted-chrome-extensions (#6720)
• Restrict the usb_devices pack to Posix (#6739)
• Add Reptile rootkit to ossec-rootkit pack (#6703)

4.5.1

Git Commits

Under the Hood improvements

• Improve carver tests by faking postCarve (#6659)
• Emit an error during carving, if the carve SQL function is disabled (#6658)
• Update carves specs to allow full scan (#6657)
• Update carves table to use JSON (#6656)
• Improve performance and accuracy of Windows registry querying (#6647)
• Refactor ephemeral database plugin into core and simplify tests (#6648)

Table Changes

• Support for Office MRU (most recently used) entries (#6587)
• Implement configurable timeout through WHERE clause on curl_certificate (#6641)
• Add atom_packages table spec to window (#6649)
• Add signature information to authenticode table on windows (#6677)
• Add additional AWS regions (#6666)

Bug Fixes

• Fix container overflow in curl_certificate (#6664)
• Fix handling of invalid array bound error with EvtNext function (#6660)
• Fix wmi_bios_info table searching (#5246)
• Fix image column within drivers table on Windows (#6652)
• Fix windows dirPathsAreEqual to use the documented way (#6690)
• Fix incorrect stat() return checking within process_events (#6694)
• Always flush stdout when called with --help (#6693)

Documentation

• Document max scheduled query interval (#6683)
• Update documentation around build steps (#6681)
• Documentation copy editing (#6676, #6665, #6662)
• Add 4.5.0 CHANGELOG (#6646)
• Add 4.5.1 CHANGELOG (#6692)

Build

• Improve flaky python test handling (#6654)
• Restore test_osqueryi (#6631)
• Limit osqueryd CPU usage to 20% in systemd unit file (#6644)
• Improve flaky test_osqueryi (#6688)
• Add cppcheck support to macOS (#6685)

Hardening

• Add exception catching for table execution (#6689)

4.5.0

Git Commits

We would like to thank all of the contributors working on bootstrapping the ARM64/AARCH64 support and Windows 32bit support. Additionally, we want to thank those working on Unicode support and all the bug fixes, documentation improvements, and new features. Thank you! 👏

New Features

• ARM64/AARCH64 beta support for Linux (#6612)
• Windows 32bit support (#6543)
• Fix buildup of RocksDB SST files (#6606)

Under the Hood improvements

• Remove selectAllFrom from Linux process_events callback (#6638)
• Remove database read only concept (#6637)
• Move database initialization retry logic into DB API (#6633)
• Move osquery/include files into respective CMake targets (#6557)
• Memoize EventFactory::getType (#6555)
• Update schedule counter behavior (#6223)
• Define UNICODE and _UNICODE preprocessors for windows (#6338)
• Add WMI utility function to convert datetime to FILETIME (#5901)
• Move osquery shutdown logic outside of Initializer (#6530)

Table Changes

• Support for Windows Background Activity Moderator (#6585)
• Add apparmor_events table to Linux (#4982)
• Add sigurl column to get YARA signatures from an HTTPS server (#6607)
• Add sigrules column to pass YARA signatures within queries (#6568)
• Add non-evented table for querying windows_event_log (#6563)
• Improve chassis_types and security_breach columns within chassis_info (#6608)
• Fix bool type usage in powershell_events (#6584)
• Add FileVersionRaw column to file table for Windows (#5771)
• Enable YARA table on Windows (#6564)
• Add dns_cache table for Windows (#6505)
• Add support for processing KILL syscall (#6435)
• Add startup_items table for Linux (#6502)
• Add shimcache table (#6463)
• Refactor shell_history to use generators (it will use less memory) (#6541)

Bug Fixes

• Set thread names correctly on macOS and Linux (#6627)
• Apply --scheduler_timeout correctly (#6618)
• Add check for character_frequencies size (#6625)
• Fix race in removing external TablePlugins (#6623)
• Force shell to disable watchdog and logger (#6621)
• Return early within the shell if relative flags are used (#6605)
• Apply watcher delay each time the worker is started (#6604)
• Set global output function for Thrift (#6592)
• Fix incorrect readFile params in createPidFile (#6578)
• Fix call to LocalFree on deinit ptr inside getUidFromSid (#6579)
• Fix readFile to observe requested read size (#6569)
• Replace fstream within syslog_events with a custom non-blocking getline (#6539)
• Only fire events if a publisher exists (#6553)
• Fix Leak in psidToString (#6548)
• Fix memory leaks in rpm_package_files (#6544)
• Change "Symlink loop" message from warning to verbose (#6545)

Documentation

• Update process auditing docs schema link (#6645)
• Improve descriptions for the processes table (#6596)
• Replace slackin with Slack shared invite (#6617)
• Update copyright notices to osquery foundation (#6589, #6590)

Build

• Fix Windows build by removing non existing C11 conformance (#6629)
• Remove ExecStartPre from systemd service unit (#6586)
• Fix pip upgrade warning within CI (#6576)
• Detect MAJOR_IN_SYSMACROS/MKDEV for librpm in CMake (#6554)
• Add curl_certificate tests (#5281)
• Update YARA library to 4.0.2 (#6559)
• Improve testing assumptions and flush fsevents when stopping (#6552)
• Fix the test utility to allow Windows profiling (#6550)
• Support ASAN for boost coroutine2 using ucontext (#6531)
• Update instructions for CPack package building (#6529)
• Use specific RPM variables to set the package name (#6527)
• Update compiler version used to v142 within Azure (#6528)

Hardening

• Restore PIE support being dropped on Linux (#6611)

4.4.0

Git Commits

New Features / Under the Hood improvements

• Implement container access from tables on Linux (#6209, #6485)
• Update language to use 'allow list' and 'deny list' (#6489, #6487, #6488, #6493)
• macos: Automatic configuration of the OpenBSM audit rules (#6447)
• macos: Add polling to OpenBSM publisher (#6436)
• Add messages to distributed query results (#6352)
• Implement event batching support for Windows tables (#6280)

Table Changes

• Add container access to the os_version table (#6413)
• Add container access to DEB, RPM, NPM packages tables (#6414)
• Add fields auid, fs{u,g}id, s{u,g}id to auditd based tables (#6362)
• Improve apt_sources resiliency (#6482)
• Make file and hash container columns hidden (#6486)
• Add 'maintainer', 'section', 'priority' columns to deb_packages (#6442)
• Add 'vendor', 'package_group' columns to rpm_packages (#6443)
• Add 'arch' column to os_version (#6444)
• Add 'board_xxx' columns to system_info table (#6398)
• Windows: omit non-interactive sessions from logged_in_users (#6375)
• Fixes to package_bom table (#6457, #6461)
• Add chassis_info table for windows (#5282)
• Add Azure tables (#6507)

Bug Fixes

• Update hash cache inode number in query cache (#6440)
• Only explode registry key if it can be tokenized (#6474)
• Change ErrorBase::takeUnderlyingError to non const (#6483)
• Use RapidJSON to fix event format results and the Kafka Logger (#6449)
• Correct the 'cwd' and 'root' columns of processes table on Windows (#6459)
• Correct some SQLite types (#6392)
• Partial fix for md_devices issue (#6417)
• Fix the handling of empty args strings, on Windows (#6460)
• Refactor shutdown logging, and remove explicit syslog call (#6376)
• Change the Windows registry LIKE path constraint to filter recursively (#6448)
• Use sync resolve within http client (#6490)
• Fix typed_row table caching (#6508)
• Do not use system proxy for AWS local authority (#6512)
• Only populate table cache with star-like selects (#6513)

Documentation

• Update osquery security policy (#6425)
• Updating changelog for 4.3.0 release (#6387)
• Improve the new table tutorial (#6479)
• Add Auto Table Construction to docs (#6476)
• Add documentation for enabling socket_events on macOS (#6407)
• Update winbaseobj table description (#6429)
• Fixing the description of failed_login_count from account_policy_data (#6415)
• Remove references to brew in macOS install (#6494)
• Add note to bump the Homebrew cask (#6519)
• Updating docs on cpack usage to include Chocolatey (#6022)
• Changelog for 4.4.0 (#6492, #6523))

Build

• Fix Userassist.test_sanity test sometimes failing (#6396)
• Drop the facebook and source_migration layers (#6473)
• Move ssdeep-cpp to source_migration (#6464)
• Move smartmontools to source_migration (#6465)
• Build augeas from source on macOS (#6399)
• Build lldpd from source on macOS (#6406)
• Build linenoise-ng from source on macOS and Windows (#6412)
• Build sleuthkit from source on macOS (#6416)
• Build popt from source on macOS (#6409)
• Fix libelfin build on ossfuzz and LLVM/Clang 10 (#6472)
• Use the patched libelfin version (#6480)
• codegen: Port Jinja2 to Templite (#6470)
• Pass the minimum macOS SDK version to openssl only if explicitly set (#6471)
• Add git-lfs as dep for macOS build in documentation (#6384)
• Update openssl from 1.1.1f to 1.1.1g (#6432)
• Build openssl with the macOS SDK version taken from CMake (#6469)
• Do not install openssl docs (#6441)
• Update build configuration of ReadTheDocs (#6434, #6456)
• Link librdkafka on Windows (#6454)
• Build sleuthkit on Windows (#6445)
• Add nupkg cpack build option and update Windows deployment script (#6262)
• Fix rpm and deb package name format (#6468)
• Fix atom_packages, processes, rpm_packages tests (#6518)
• Fixes and cleanup for Windows compiler flags (#6521)
• Correct macOS framework linking (#6522)

Security Issues

• Disable openssl compression support (#6433)

Hardening

• Use LOAD_LIBRARY_SEARCH_SYSTEM32 for LoadLibrary (#6458)

4.3.0

Git Commits

New Features / Under the Hood improvements

• Change verbosity of scheduled query execution messages from INFO to verbose only (#6271)
• Updated the unwanted-chrome-extensions queries to include all users, not the osquery process owner only (#6265)
• Check for errors in the return status of the extension tables and report them (#6108)
• First steps to properly support UTF8 strings on Windows (#6190)
• Display the undelying API error string when udev monitoring fails (#6186)
• Add the path column to the ATC generate specs (#6278)
• Add Kafka support to Microsoft Windows (#6095)
• Log a warning message if osquery fails to get the service description on Microsoft Windows (#6281)
• Make AWS kinesis status logging configurable (#6135)
• Add an integration test for the disk_info table (#6323)
• Use -1 for missing ppid in the process_events table (#6339)
• Remove error when converting empty numeric rows (#6371)
• Change verbosity from ERROR to INFO of access failures to system processes on Microsoft Windows (#6370)
• Make possible to get verbose messages from the dispatcher service management on Microsoft Windows too (#6369)

Build

• Fix codegen template for extension group (#6244)
• Update SQLite from 3.30.1-1 to 3.31.1 (#6252)
• Update the osquery-toolchain to version 1.1.0 which uses LLVM/Clang 9.0.1 (#6315)
• Update openssl to version 1.1.1f (#6302, #6359)
• Simplify formula-based third party libraries build (#6303)
• Removed the Buck build system (#6361)

Bug Fixes

• Fix CFNumber conversion when the type was a Float64/32 instead of a Double (#6273)
• Fix duplicate results being returned by the chrome_extensions table (#6277)
• Fix flaky ProcessOpenFilesTest.test_sanity (#6185)
• Fix the --database_dump flag for RocksDB not outputting anything (#6272)
• Fix the pci_devices table pci ids extraction in non-existing paths (#6297)
• Fix parsing an invalid decorators config (#6317)
• Fix flaky TLSConfigTests.test_runner_and_scheduler (#6308)
• Fix chromeExtensions.test_sanity (#6324)
• Fix broken Unicode filename searches on Microsoft Windows (#6291)
• Fix a use-after-free when sqlite attempts to access the entire rows data at the end of a query (#6328)
• Keep proc instance for test_base and test_osqueryd (#6335)
• Fix osquery not exiting when given check or dump requests (#6334)
• Fix process table cmdline parsing (#6340)
• Fix a crash when parsing files with libmagic (#6363)
• Fix a sporadic readFile API failure when using non-blocking I/O (#6368)
• Fix the MSI package not always installing in the system drive by default (#6379)
• Ensure the extensions uuid is never 0 (#6377)
• Fix a race condition making the watcher act as a worker on Microsoft Windows (#6372)
• Fix extensions tables detaching which was sometimes failing (#6373)
• Fix an issue with extensions re-registration (#6374)
• Fix a crash due to a race condition in accessing the iokit port on Darwin (Apple OS X) (#6380)

Hardening

• Limit SQL functions regex_match and regex_split regex size (#6267)
• Prevent a stack overflow when parsing deeply nested configs (#6325)

Table Changes

• Added table chrome_extension_content_scripts to All Platforms (#6140)
• Added table docker_container_fs_changes to POSIX-compatible Plaforms (#6178)
• Added table windows_security_center to Microsoft Windows (#6256)
• Added many new tables to Linux to query lxd (#6249)
• Added table screenlock to Darwin (Apple OS X) (#6243)
• Added table userassist to Microsoft Windows (#5539)
• Added column status (TEXT) to table deb_packages (#6341)
• Added many new columns to the curl_certificate table (#6176)
• Added table socket_events to Darwin (Apple OS X) (#6028)
• Added table hvci_status, previously inadvertly left out from the build, to Microsoft Windows (#6378)

4.2.0

Git Commits

New Features / Under the Hood improvements

• TLS Testing infrastructure has been overhauled (#6170)
• Boost regex has been replaced with std (#6236)
• community_id_v1 added as a SQL function (#6211)

Build

• Fix format checking on Windows (#6188)
• Fix format folder exclusions for build checks (#6201)
• Fix the linking for extensions in build (#6219)
• Fix build to include windows optional features table (#6207)

Security Issues

• [CVE-2020-1887] osquery does not properly verify the SNI hostname (#6197)

Bug Fixes

• Carver no longer returns empty carves for hidden files (#6183)
• Address a race in the Dispatcher logic (#6145)
• Fix validation in 'last' table (#6147)
• Fix flaky logger testing (#6171)
• Fix JSON format assumptions in file_paths parsing (#6159)
• Fix windows WMI BSTR to be wstrings (#6175)
• Fix windows string <-> wstring conversion functions (#6187)
• Enable more intelligent path expansion on Windows (#6153)
• Fix heap buffer overflow in callDoubleFunc and powerFunc (#6225)

Table Changes

• Added table firefox_addons to All Platforms (#6200)
• Added table ssh_configs to All Platforms (#6161)
• Added table user_ssh_keys to All Platforms (#6161)
• Added table mdls to Darwin (Apple OS X) (#4825)
• Added table hvci_status to Microsoft Windows (#5426)
• Added table ntfs_journal_events to Microsoft Windows (#5371)
• Added table docker_image_layers to POSIX-compatible Plaforms (#6154)
• Added table process_open_pipes to POSIX-compatible Plaforms (#6142)
• Added table apparmor_profiles to Ubuntu, CentOS (#6138)
• Added table selinux_settings to Ubuntu, CentOS (#6118)
• Added column lock_status (INTEGER_TYPE) to table bitlocker_info (#6155)
• Added column percentage_encrypted (INTEGER_TYPE) to table bitlocker_info (#6155)
• Added column version (INTEGER_TYPE) to table bitlocker_info (#6155)
• Added column optional_permissions (TEXT_TYPE) to table chrome_extensions (#6115)
• Removed table firefox_addons from POSIX-compatible Plaforms (#6200)
• Removed table ssh_configs from POSIX-compatible Plaforms (#6161)
• Removed table user_ssh_keys from POSIX-compatible Plaforms (#6161)

4.1.2

Git Commits

New Features / Under the Hood improvements

• Add more tests throughout the codebase (#5908), (#6071), (#6126)
• The chrome_extensions table now supports Chromium and Brave (#6126)

Build

• Require Python 3.5 and greater (#6081), (#6120)
• Prepare Python tests for CI (lots of effort!) (#6068)
• Restore osqueryd integration test (#6116)

Bug Fixes

• Continue to use com.facebook.osquery.plist for Launch Daemon configuration (#6093)
• Update systemd service to use KillMode=control-group (#6096)
• RPM and DEB packages both have post-install scripts to reload systemd (#6097)
• Update Windows package build script to include cert bundle (#6114)
• Update table specs to fix constraints passing (#6103), (#6104), (#6105), (#6106), (#6122)

Table Changes

• Added tables azure_instance_tags and azure_instance_metadata to Linux and Microsoft Windows (#5434)
• Added column install_time (INTEGER_TYPE) to table rpm_packages (#6113)
• Added column bsd_flags (TEST_TYPE) to table file on Darwin (#5981)

4.1.1

Git Commits

New Features / Under the Hood improvements

• Improve nvram table to use input variable names (#6053)
• Improve apt_sources source detection (#6047)
• Change atom_packages to use user constraints (#6052)
• Re-enable required-column warning messages (#6038)

Build

• Migrate several libraries to the CMake source layer (#5902), (#6023)
• Update SQLite from 3.29.0-3 to 3.30.1-1 (#6020)
• Recommend building with MacOS 10.11 SDK (#6000)

Bug Fixes

• Fix Linux audit incorrect read and handle leak (#5959)
• Change "logNumericsAsNumbers" to "numerics" logger top-level key (#6002)
• Restore INDEX behavior for extensions (#6006)
• Fix potential JSON parsing issues in ATC plugin (#6029)
• Avoid scanning special files with YARA (#5971)
• Fix use-after-move in YARA subscriber (#6054)
• Handle relative redirects in internal HTTP clients (#6049)
• Apply options config parsing before others (#6050)

Table Changes

• Added table windows_optional_features to Microsoft Windows #5991)

4.1.0

Git Commits

New Features / Under the Hood improvements

• Restore extension SDK and build support (#5851)
• Documentation improvements (#5860), (#5852), (#5912), (#5954)
• Add more tests throughout the codebase (#5837), (#5832), (#5857), (#5864), (#5855), (#5869), (#5871), (#5885), (#5903), (#5879), (#5914), (#5941), (#5957)
• Allow configuration more Linux Audit settings using flags (#5953)
• Add logger_tls_max_lines flag (#5956)
• Add AWS Session Token support (#5944)

Build

• Lots of work on CPack-based packaging (#5809), (#5822), (#5823), (#5827), (#5780), (#5850), (#5843), (#5881), (#5825), (#5940), (#5951), (#5936)
• Lots of work porting Python2 to Python3 (#5846)
• Upgrade OpenSSL to 1.0.2t on all platforms (#5928)
• Use SQLite 3.29.0 on Windows and macOS (#5810)
• Use aws-sdk-cpp source-builds on Windows and macOS (#5889)
• Add various code quality checks and utilities (#5834), (#5730), (#5872)

Hardening

• Restore fuzzing harness and use oss-fuzz (#5844), (#5886), (#5910), (#5915), (#5923), (#5955), (#5963)
• Use newer RapidJSON and switch to safer iterative parsing (#5893), (#5913)

Bug Fixes

• Set Windows MSI ErrorControl to normal instead of critical (#5818)
• Wrap flagfile with quotes for Windows install flag (#5824)
• Improve submodule usages in CMake (#5850), (#5880), (#5892), (#5897), (#5907)
• Improve locking support in internal APIs (#5841), (#5906), (#5943), (#5944)
• Fixes for macOS application layer firewall tables (#5378)
• Fixes within BPF event tables (#5874)
• Refactor and improve PCI device tables on Linux (#5446)
• Implement PID indexing on Windows processes table (#5919)
• Improve WHERE IN() performance (#5924), (#5938)
• Improve the internal HTTP client (#5891), (#5946), (#5947)
• Fix Windows version codename lookup (#5887)

Table Changes

• Added table alf_services to Darwin (Apple OS X) (#5378)
• Added table connectivity to Microsoft Windows (#5500)
• Added table default_environment to Microsoft Windows (#5441)
• Added table windows_security_products to Microsoft Windows (#5479)
• Added column platform_mask (INTEGER_TYPE) to table osquery_info (#5898)

4.0.2

This release fixes crashes identified in 4.0.1. There are no changes in functionality.

Git Commits

Bug Fixes

• Fix configuration of AWS libraries to address crash in Linux (#5799)
• Remove RocksDB optimization causing crash (#5797)

4.0.1

This release has two major focuses. It is the first release since osquery transitioned to a Linux Foundation project.

It features a heavily reworked build system. This aims to provide flexibility and stability.

Git Commits

New Features / Under the Hood improvements

• Linux Audit process_events Implement support for fork/vfork/clone/execveat (#5701)
• New SQLite function regex_match to match across columns (#5444)
• LRU cache for syscall tracing (#5521)
• Basic tracing via eBPF on Linux (#5403, #5386, #5384)
• Experimental kill and setuid syscall tracing in Linux via eBPF (#5519)
• New eventing (ev2) framework (#5401)
• Improved table performance profiles (#5187)
• macOS query pack: detect SearchAwesome malware (#5713)
• macOS query pack: detect when a process is tapping keyboard event (#5345)

Build

• Refactor CMake build (#5604, #5627, #5630, (#5618), (#5619))
• Refactor third-party libraries to build from source on Linux (#5706)
• Add Azure Pipelines support for CI/CD (#5604, #5632, #5626, #5613, #5607, #5673, #5610)
• Add Buck as a build system (971bee44)
• Use urllib2 to automatically handle HTTP 301/302 redirections (#5612)
• Update MSI package to install to Program Files on Windows (#5579)
• Linux custom toolchain integration (#5759)

Hardening

• Link binaries with Full RELRO on Linux (#5748)
• Remove FTS features from SQLite (#5703, #5702)
• Fix SQLite API usage errors (#5551)
• Fix issues reported by ASAN (#5665)
• Handle bad FDs in md_tables (#5553)
• Fix lock resource leak in events/syslog (#5552)
• Fix memory leak in macOS keychain_items and extended_attributes tables (#5550, #5538)
• Fix memory leak in genLoggedInUsers (Windows). Update WTSFreeMemoryEx to WTSFreeMemory (#5642)
• Fix potential null dereferences in smbios_tables (#5332)
• Fix osquery exiting with wrong status (3824c2e6)
• Add additional install and uninstall flag incompatibility check (85eb77a0)
• Fix warning with constants initialisation in magic (2a624f2f)
• Fix sign compare warning in file_compression (b93069b3)
• Refactored logical_drives table on Windows (#5400)
• Refactored core/windows/wmi to use smart pointers (#5492)
• Fixed various potential crashes in the virtual table implementation (6ade85a5)
• Increase the amount of MaxRecvRetries for Thrift sockets (#5390)

Bug Fixes

• Fix the reading of the serial of a certificate (little-endian big int) (#5742)
• Fix bugs and update pathname variables in MSI package build script (#5733)
• Fix registry table exception closing an uninitialized key handle (#5718)
• Config views are now recreated on startup (#5732)
• Change MSI Service Error handling on Windows (#5467)
• Allow mounting SQLite DBs using WAL journaling with ATC (#5525, #5633)
• Fix mount table interacting with direct autofs (#5635)
• Fix HTTP Host Header to include port (#5576)
• Various fixes to the Windows certificates table and expansion to include Personal certificates (#5697), (#5696), (#5640), (#5631)
• Add optimization back to macOS users and groups (#5684)
• Do not return a row for macOS battery if no data is present (#5650)
• Fix several integer conversions in process_ops (#5614)
• Include weekends on the kernel_panics table (#5298)
• Fix key_strength bug for Windows certificates table (#5304)
• The interface column of routes table could be empty on Windows (bcf0ab8e)
• The name column of programs table could be empty on Windows (7bceba4b)
• Fix disable_watcher flag (08dc11b7)
• Populate path column correctly in firefox_addons table (#5462)
• Fix numeric monitoring plugin not being registered (#5484)
• Fix wrong error code returned when querying the Windows registry (#5621)
• Fix logical_drives boot partition detection (#5477)
• Replace sync calls by async within the HTTP client implementation (#5606)
• Fix RocksDB crash related to OptimizeForSmallDb (a31d7582)
• Fix bug in table column data validator (e3037331)
• Fix random port problem (a32ed7c4)
• Refactor battery table and return information even if advanced information is missing (6a64e353)

Table Changes

• Added table ibridge_info on macOS (Notebooks only) (#5707)
• Added table running_apps on macOS (#5216)
• Added table atom_packages on macOS and Linux (6d159d40)
• Remove EC2 tables on Windows (#5657)
• Add column win_timestamp to time table on Windows (3bbe6c51)
• Add column is_hidden to users and groups table on macOS (#5368)
• Add column profile to chrome_extensions table (#5213)
• Add column epoch to rpm_packages table on Linux (#5248)
• Add column sid to logged_in_users table on Windows (#5454)
• Add column registry_hive to logged_in_users table on Windows (#5454)
• Add column sid to certificates table on Windows (#5631)
• Add column store_location to certificates table on Windows (#5631)
• Add column store to certificates table on Windows (#5631)
• Add column username to certificates table on Windows (#5631)
• Add column store_id to certificates table on Windows (#5631)
• Add column product_version to file table on Windows (#5431)
• Add column source to sudoers table on POSIX systems (#5350)