Skip to content

Commit dc9695b

Browse files
authored
Update standalone-with-crucible docs (#514)
1 parent f78820b commit dc9695b

File tree

1 file changed

+63
-20
lines changed

1 file changed

+63
-20
lines changed

docs/standalone-with-crucible.md

+63-20
Original file line numberDiff line numberDiff line change
@@ -26,26 +26,31 @@ TODO: flesh out more of these differences, and maybe capture them in a
2626
higher-level README.
2727

2828
## Requirements
29-
30-
TODO: document the following
31-
32-
- How to build each of propolis and crucible
33-
- Assuming we are running on bench gimlets, which files to copy
34-
* propolis-standalone
35-
* crucible release binaries
36-
- target/release/dsc
37-
- target/release/crucible-downstairs
29+
### Building `propolis-standalone`
30+
- Clone this repository on an illumos box (e.g. `atrium`)
31+
- In that folder, run
32+
`cargo build --release -ppropolis-standalone --features=crucible`
33+
- This will produce a `propolis-standalone` binary in `target/release/`
34+
- Copy this binary to your target Gimlet
35+
36+
### Building `crucible`
37+
- Clone [`oxidecomputer/crucible`](https://github.com/oxidecomputer/crucible) on
38+
an illumos box (e.g. `atrium`)
39+
- In that folder, run `cargo build --release -pcrucible-downstairs -pdsc`
40+
- This will produce `crucible-downstairs` and `dsc` binaries in `target/release`
41+
- Copy those files to your target Gimlet
42+
43+
### VM stuff
44+
See the [`propolis-standalone` README](../bin/propolis-standalone/README.md)
45+
for details on how to get
3846
* VM Image file
3947
* VM OVMF file
40-
* standalone.toml
4148

42-
## Instructions
49+
Copy those files to your target Gimlet.
4350

44-
### Copy over the required files.
45-
46-
TODO:
51+
## Instructions
4752

48-
### Onetime setup on the gimlet.
53+
### Onetime setup on the Gimlet.
4954

5055
Setup for a virtual NIC to be used by the VM.
5156

@@ -56,11 +61,44 @@ dladm create-vnic -t -l igb0 -m 02:08:20:ac:e9:16 vnic_prop0
5661
Setup of a zpool on three SSDs.
5762

5863
Crucible downstairs runs on top of a filesystem (ZFS in our case).
59-
On your bench gimlet, you should select three NVMe disks, and create a zpool
64+
On your bench Gimlet, you should select three NVMe disks, and create a zpool
6065
on each of them. You can use an existing zpool.
6166

67+
If you're creating new zpools, start by running `format` to list disk names:
6268
```
63-
TODO: zfs pool create commands
69+
BRM42220012 # format
70+
Searching for disks...done
71+
72+
73+
AVAILABLE DISK SELECTIONS:
74+
0. c1t00A0750130082207d0 <NVMe-Micron_7300_MTFDHBG1T9TDF-95420260-1.75TB>
75+
/pci@0,0/pci1de,fff9@1,3/pci1344,3100@0/blkdev@w00A0750130082207,0
76+
1. c2t0014EE81000BC481d0 <NVMe-WUS4C6432DSP3X3-R2210000-2.91TB>
77+
/pci@0,0/pci1de,fff9@3,2/pci1b96,0@0/blkdev@w0014EE81000BC481,0
78+
2. c3t0014EE81000BC783d0 <NVMe-WUS4C6432DSP3X3-R2210000-2.91TB>
79+
/pci@0,0/pci1de,fff9@3,3/pci1b96,0@0/blkdev@w0014EE81000BC783,0
80+
3. c4t0014EE81000BC78Fd0 <NVMe-WUS4C6432DSP3X3-R2210000-2.91TB>
81+
/pci@0,0/pci1de,fff9@3,4/pci1b96,0@0/blkdev@w0014EE81000BC78F,0
82+
4. c5t0014EE81000BC37Dd0 <NVMe-WUS4C6432DSP3X3-R2210000-2.91TB>
83+
/pci@38,0/pci1de,fff9@1,2/pci1b96,0@0/blkdev@w0014EE81000BC37D,0
84+
5. c6t0014EE81000BC28Ad0 <NVMe-WUS4C6432DSP3X3-R2210000-2.91TB>
85+
/pci@38,0/pci1de,fff9@1,3/pci1b96,0@0/blkdev@w0014EE81000BC28A,0
86+
6. c7t00A0750130082248d0 <NVMe-Micron_7300_MTFDHBG1T9TDF-95420260-1.75TB>
87+
/pci@38,0/pci1de,fff9@3,3/pci1344,3100@0/blkdev@w00A0750130082248,0
88+
7. c8t0014EE81000BC39Bd0 <NVMe-WUS4C6432DSP3X3-R2210000-2.91TB>
89+
/pci@ab,0/pci1de,fff9@1,1/pci1b96,0@0/blkdev@w0014EE81000BC39B,0
90+
8. c9t0014EE81000BC3C8d0 <NVMe-WUS4C6432DSP3X3-R2210000-2.91TB>
91+
/pci@ab,0/pci1de,fff9@1,2/pci1b96,0@0/blkdev@w0014EE81000BC3C8,0
92+
9. c10t0014EE81000BC4CCd0 <NVMe-WUS4C6432DSP3X3-R2210000-2.91TB>
93+
/pci@ab,0/pci1de,fff9@1,3/pci1b96,0@0/blkdev@w0014EE81000BC4CC,0
94+
10. c11t0014EE81000BC786d0 <NVMe-WUS4C6432DSP3X3-R2210000-2.91TB>
95+
/pci@ab,0/pci1de,fff9@1,4/pci1b96,0@0/blkdev@w0014EE81000BC786,0`
96+
Specify disk (enter its number): ^C
97+
```
98+
99+
Then, create zpools with your desired serial names, e.g.
100+
```
101+
zpool create -f -o ashift=12 -O atime=off -m /pool/disk0 cru0 c1t00A0750130082207d0
64102
```
65103

66104
On each zpool, create a directory where the crucible downstairs will live:
@@ -102,6 +140,9 @@ For 4096 byte blocks, 16384 is the extent size.
102140
--region-dir /pool/disk2/region
103141
```
104142

143+
(modify the `dsc` and `crucible-downstairs` paths based on where you put those
144+
binaries)
145+
105146
### Run the three downstairs
106147

107148
Once the regions are created, you can start the three downstairs using the
@@ -115,10 +156,12 @@ Once the regions are created, you can start the three downstairs using the
115156
--region-dir /pool/disk2/region
116157
```
117158

118-
### Start propolis-standalone
159+
### Start `propolis-standalone`
160+
161+
To start `propolis-standalone`, you'll need a configuration file. The specifics
162+
will depend on file paths, image type, etc.
119163

120-
standalone toml example
121-
This example file assumes you have used the above settings for block
164+
Here's an example TOML file, assuming you have used the above settings for block
122165
size, extent size, and extent count.
123166

124167
```toml

0 commit comments

Comments
 (0)