Skip to content

Commit 3cdb2e6

Browse files
chklauserchklauser
committed
Automatically use CMake for Windows builds
This PR changes the `rdkafka-sys/build.rs` script to automatically choose CMake if the target platform is Windows. The `cmake-build` cargo feature no longer has an effect on Windows. It can still be used to select the CMake build for other targets. The official rdkafka build system remains the default for other targets. This solves a number of problems: - Projects using `rdkafka`/`rdkafka-sys` no longer have to conditionally enable `cmake-build` when they reference the crates just to have something that compiles across platforms. - JetBrains RustRover + rust-analyzer fail to initialize on Windows because the `rdkafka-sys/build.rs` script fails. In RustRover, even if you otherwise enable the `cmake-build` feature, there is an early "disovery" execution of the project that uses default features and fails miserably. - People evaluating the crate have a better experience if they happen to run Windows
1 parent e69c2aa commit 3cdb2e6

File tree

8 files changed

+405
-393
lines changed

8 files changed

+405
-393
lines changed

‎.github/workflows/ci.yml

+3-2
Original file line numberDiff line numberDiff line change
@@ -28,8 +28,9 @@ jobs:
2828
include:
2929
- os: macos-11.0
3030
- os: windows-2019
31-
features: cmake-build,libz-static,curl-static
32-
rdkafka-sys-features: cmake-build,libz-static,curl-static
31+
# The Windows build should automatically use CMake
32+
features: libz-static,curl-static
33+
rdkafka-sys-features: libz-static,curl-static
3334
- os: ubuntu-20.04
3435
features: tracing
3536
- os: ubuntu-20.04

‎README.md

+290-289
Large diffs are not rendered by default.

‎changelog.md

+2
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,8 @@ See also the [rdkafka-sys changelog](rdkafka-sys/changelog.md).
44

55
## Unreleased
66

7+
* Automatically use CMake for Windows targets
8+
79
## 0.36.2 (2024-01-16)
810

911
* Update `BaseConsumer::poll` to return `None` when handling rebalance

‎rdkafka-sys/Cargo.toml

+6
Original file line numberDiff line numberDiff line change
@@ -26,6 +26,12 @@ sasl2-sys = { version = "0.1.6", optional = true }
2626
pkg-config = "0.3.9"
2727
cmake = { version = "0.1.0", optional = true }
2828

29+
# Windows only supports the CMake build. As we can't conditionally
30+
# make cmake-build a default feature only on some platforms, we instead
31+
# have to make the cmake dependency required.
32+
[target.'cfg(target_os = "windows")'.build-dependencies]
33+
cmake = { version = "0.1.0", optional = false }
34+
2935
[lib]
3036
name = "rdkafka_sys"
3137
path = "src/lib.rs"

‎rdkafka-sys/README.md

+95-94
Original file line numberDiff line numberDiff line change
@@ -1,94 +1,95 @@
1-
# rdkafka-sys
2-
3-
Low level bindings to [librdkafka](https://github.com/edenhill/librdkafka),
4-
a C library for the [Apache Kafka] protocol with producer, consumer, and
5-
admin clients.
6-
7-
For a safe wrapper, see the [rdkafka] crate.
8-
9-
## Version
10-
11-
The rdkafka-sys version number is in the format `X.Y.Z+RX.RY.RZ`, where
12-
`X.Y.Z` is the version of this crate and follows SemVer conventions, while
13-
`RX.RY.RZ` is the version of the bundled librdkafka.
14-
15-
Note that versions before v2.0.0+1.4.2 did not follow this convention, and
16-
instead directly correspond to the bundled librdkafka version.
17-
18-
## Build
19-
20-
### Known issues
21-
22-
* When any of librdkafka's optional dependencies are enabled, like libz or
23-
OpenSSL, if you have multiple versions of that library installed upon your
24-
system, librdkafka's build system may disagree with Cargo about which
25-
version of the library to use! **This can result in subtly broken
26-
builds,** if librdkafka compiles against the headers for one version but
27-
Cargo links against a different version. For complete confidence when
28-
building release binaries, use an environment like a Docker container or a
29-
chroot jail where you can guarantee that only one version of each
30-
dependency is present. The current design of Cargo unfortunately makes
31-
this nearly impossible to fix.
32-
33-
* Windows is only supported when using the CMake build system via the
34-
`cmake-build` Cargo feature.
35-
36-
### Features
37-
38-
By default a submodule with the librdkafka sources will be used to compile
39-
and statically link the library.
40-
41-
The **`dynamic-linking`** feature can be used to link rdkafka to a locally
42-
installed version of librdkafka: if the feature is enabled, the build script
43-
will use `pkg-config` to check the version of the library installed in the
44-
system, and it will configure the compiler to dynamically link against it.
45-
The system version of librdkafka must exactly match the version of
46-
librdkafka bundled with this crate.
47-
48-
The **`cmake-build`** feature builds librdkafka with its [CMake] build
49-
system, rather than its default [mklove]-based build system. This feature
50-
requires that CMake is installed on the build machine.
51-
52-
The following features directly correspond to librdkafka features (i.e.,
53-
flags you would pass to `configure` if you were compiling manually).
54-
55-
* The **`ssl`** feature enables SSL support. By default, the system's
56-
OpenSSL library is dynamically linked, but static linking of the version
57-
bundled with the [openssl-sys] crate can be requested with the
58-
`ssl-vendored` feature.
59-
* The **`gssapi`** feature enables SASL GSSAPI support with Cyrus
60-
libsasl2. By default the system's libsasl2 is dynamically linked, but
61-
static linking of the version bundled with the [sasl2-sys] crate can be
62-
requested with the `gssapi-vendored` feature.
63-
* The **`libz`** feature enables support for zlib compression. This
64-
feature is enabled by default. By default, the system's libz is
65-
dynamically linked, but static linking of the version bundled with the
66-
[libz-sys] crate can be requested with the `libz-static` feature.
67-
* The **`curl`** feature enables the HTTP client via curl. By default, the
68-
system's curl is dynamically linked, but static linking of the version
69-
bundled with the [curl-sys] create can be requested with the
70-
`curl-static` feature.
71-
* The **`zstd`** feature enables support for ZSTD compression. By default,
72-
this builds and statically links the version bundled with the [zstd-sys]
73-
crate, but dynamic linking of the system's version can be requested with
74-
the `zstd-pkg-config` feature.
75-
* The **`external-lz4`** feature statically links against the copy of
76-
liblz4 bundled with the [lz4-sys] crate. By default, librdkafka
77-
statically links against its own bundled version of liblz4. Due to
78-
limitations with lz4-sys, it is not yet possible to dynamically link
79-
against the system's version of liblz4.
80-
81-
All features are disabled by default unless noted otherwise above. The build
82-
process is defined in [`build.rs`].
83-
84-
[`build.rs`]: https://github.com/fede1024/rust-rdkafka/tree/master/rdkafka-sys/build.rs
85-
[Apache Kafka]: https://kafka.apache.org
86-
[CMake]: https://cmake.org
87-
[libz-sys]: https://crates.io/crates/libz-sys
88-
[curl-sys]: https://crates.io/crates/curl-sys
89-
[lz4-sys]: https://crates.io/crates/lz4-sys
90-
[mklove]: https://github.com/edenhill/mklove
91-
[openssl-sys]: https://crates.io/crates/openssl-sys
92-
[rdkafka]: https://docs.rs/rdkafka
93-
[sasl2-sys]: https://docs.rs/sasl2-sys
94-
[zstd-sys]: https://crates.io/crates/zstd-sys
1+
# rdkafka-sys
2+
3+
Low level bindings to [librdkafka](https://github.com/edenhill/librdkafka),
4+
a C library for the [Apache Kafka] protocol with producer, consumer, and
5+
admin clients.
6+
7+
For a safe wrapper, see the [rdkafka] crate.
8+
9+
## Version
10+
11+
The rdkafka-sys version number is in the format `X.Y.Z+RX.RY.RZ`, where
12+
`X.Y.Z` is the version of this crate and follows SemVer conventions, while
13+
`RX.RY.RZ` is the version of the bundled librdkafka.
14+
15+
Note that versions before v2.0.0+1.4.2 did not follow this convention, and
16+
instead directly correspond to the bundled librdkafka version.
17+
18+
## Build
19+
20+
### Known issues
21+
22+
* When any of librdkafka's optional dependencies are enabled, like libz or
23+
OpenSSL, if you have multiple versions of that library installed upon your
24+
system, librdkafka's build system may disagree with Cargo about which
25+
version of the library to use! **This can result in subtly broken
26+
builds,** if librdkafka compiles against the headers for one version but
27+
Cargo links against a different version. For complete confidence when
28+
building release binaries, use an environment like a Docker container or a
29+
chroot jail where you can guarantee that only one version of each
30+
dependency is present. The current design of Cargo unfortunately makes
31+
this nearly impossible to fix.
32+
33+
* Windows builds always use the CMake build system, regardless of whether
34+
the `cmake-build` Cargo feature is enabled or not. This is because Windows
35+
is only supported with CMake as the build system.
36+
37+
### Features
38+
39+
By default a submodule with the librdkafka sources will be used to compile
40+
and statically link the library.
41+
42+
The **`dynamic-linking`** feature can be used to link rdkafka to a locally
43+
installed version of librdkafka: if the feature is enabled, the build script
44+
will use `pkg-config` to check the version of the library installed in the
45+
system, and it will configure the compiler to dynamically link against it.
46+
The system version of librdkafka must exactly match the version of
47+
librdkafka bundled with this crate.
48+
49+
The **`cmake-build`** feature builds librdkafka with its [CMake] build
50+
system, rather than its default [mklove]-based build system. This feature
51+
requires that CMake is installed on the build machine.
52+
53+
The following features directly correspond to librdkafka features (i.e.,
54+
flags you would pass to `configure` if you were compiling manually).
55+
56+
* The **`ssl`** feature enables SSL support. By default, the system's
57+
OpenSSL library is dynamically linked, but static linking of the version
58+
bundled with the [openssl-sys] crate can be requested with the
59+
`ssl-vendored` feature.
60+
* The **`gssapi`** feature enables SASL GSSAPI support with Cyrus
61+
libsasl2. By default the system's libsasl2 is dynamically linked, but
62+
static linking of the version bundled with the [sasl2-sys] crate can be
63+
requested with the `gssapi-vendored` feature.
64+
* The **`libz`** feature enables support for zlib compression. This
65+
feature is enabled by default. By default, the system's libz is
66+
dynamically linked, but static linking of the version bundled with the
67+
[libz-sys] crate can be requested with the `libz-static` feature.
68+
* The **`curl`** feature enables the HTTP client via curl. By default, the
69+
system's curl is dynamically linked, but static linking of the version
70+
bundled with the [curl-sys] create can be requested with the
71+
`curl-static` feature.
72+
* The **`zstd`** feature enables support for ZSTD compression. By default,
73+
this builds and statically links the version bundled with the [zstd-sys]
74+
crate, but dynamic linking of the system's version can be requested with
75+
the `zstd-pkg-config` feature.
76+
* The **`external-lz4`** feature statically links against the copy of
77+
liblz4 bundled with the [lz4-sys] crate. By default, librdkafka
78+
statically links against its own bundled version of liblz4. Due to
79+
limitations with lz4-sys, it is not yet possible to dynamically link
80+
against the system's version of liblz4.
81+
82+
All features are disabled by default unless noted otherwise above. The build
83+
process is defined in [`build.rs`].
84+
85+
[`build.rs`]: https://github.com/fede1024/rust-rdkafka/tree/master/rdkafka-sys/build.rs
86+
[Apache Kafka]: https://kafka.apache.org
87+
[CMake]: https://cmake.org
88+
[libz-sys]: https://crates.io/crates/libz-sys
89+
[curl-sys]: https://crates.io/crates/curl-sys
90+
[lz4-sys]: https://crates.io/crates/lz4-sys
91+
[mklove]: https://github.com/edenhill/mklove
92+
[openssl-sys]: https://crates.io/crates/openssl-sys
93+
[rdkafka]: https://docs.rs/rdkafka
94+
[sasl2-sys]: https://docs.rs/sasl2-sys
95+
[zstd-sys]: https://crates.io/crates/zstd-sys

‎rdkafka-sys/build.rs

+3-4
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,6 @@
11
use std::borrow::Borrow;
22
use std::env;
33
use std::ffi::OsStr;
4-
#[cfg(feature = "cmake-build")]
5-
use std::fs;
64
use std::path::{Path, PathBuf};
75
use std::process::{self, Command};
86

@@ -88,7 +86,7 @@ fn main() {
8886
}
8987
}
9088

91-
#[cfg(not(feature = "cmake-build"))]
89+
#[cfg(all(not(feature = "cmake-build"), not(target_os = "windows")))]
9290
fn build_librdkafka() {
9391
let mut configure_flags: Vec<String> = Vec::new();
9492

@@ -200,8 +198,9 @@ fn build_librdkafka() {
200198
println!("cargo:root={}", out_dir);
201199
}
202200

203-
#[cfg(feature = "cmake-build")]
201+
#[cfg(any(feature = "cmake-build", target_os = "windows"))]
204202
fn build_librdkafka() {
203+
use std::fs;
205204
let mut config = cmake::Config::new("librdkafka");
206205
let mut cmake_library_paths = vec![];
207206

‎rdkafka-sys/src/lib.rs

+3-2
Original file line numberDiff line numberDiff line change
@@ -28,8 +28,9 @@
2828
//! dependency is present. The current design of Cargo unfortunately makes
2929
//! this nearly impossible to fix.
3030
//!
31-
//! * Windows is only supported when using the CMake build system via the
32-
//! `cmake-build` Cargo feature.
31+
//! * Windows builds always use the CMake build system, regardless of whether
32+
//! the `cmake-build` Cargo feature is enabled or not. This is because Windows
33+
//! is only supported with CMake as the build system.
3334
//!
3435
//! ### Features
3536
//!

‎src/lib.rs

+3-2
Original file line numberDiff line numberDiff line change
@@ -152,13 +152,14 @@
152152
//! * GNU `make`
153153
//! * `pthreads`
154154
//! * `zlib`: optional, but included by default (feature: `libz`)
155-
//! * `cmake`: optional, *not* included by default (feature: `cmake-build`)
155+
//! * `cmake`: optional¹, *not* included by default (feature: `cmake-build`)
156156
//! * `libssl-dev`: optional, *not* included by default (feature: `ssl`)
157157
//! * `libsasl2-dev`: optional, *not* included by default (feature: `gssapi`)
158158
//! * `libzstd-dev`: optional, *not* included by default (feature: `zstd-pkg-config`)
159159
//!
160160
//! Note that using the CMake build system, via the `cmake-build` feature, is
161-
//! encouraged if you can take the dependency on CMake.
161+
//! encouraged if you can take the dependency on CMake. ¹Windows is only supported
162+
//! via CMake, where it is the default build system.
162163
//!
163164
//! By default a submodule with the librdkafka sources pinned to a specific
164165
//! commit will be used to compile and statically link the library. The

0 commit comments

Comments
 (0)