mirror of
https://github.com/seaweedfs/seaweedfs.git
synced 2026-09-13 01:50:40 +02:00
Document both ways to create a Lance table bucket
The shell command against a running cluster, and weed mini -tableBucket at startup, which takes name[:FORMAT] entries and also reads S3_TABLE_BUCKET. Notes that mini skips a bucket in a format it does not serve rather than creating one nothing can reach.
@@ -17,6 +17,12 @@ weed shell
|
||||
> s3tables.bucket -create -name vectors -format LANCE -account 000000000000
|
||||
```
|
||||
|
||||
Or pre-create it at startup, which is one line instead of two:
|
||||
|
||||
```bash
|
||||
weed mini -tableBucket=vectors:LANCE
|
||||
```
|
||||
|
||||
## 3. Connect
|
||||
|
||||
```python
|
||||
|
||||
@@ -49,6 +49,10 @@ curl -X POST $S3_ENDPOINT/ \
|
||||
>
|
||||
> From the shell: `s3tables.bucket -create -name vectors -format LANCE -account 000000000000`
|
||||
>
|
||||
> At startup: `weed mini -tableBucket=vectors:LANCE` pre-creates buckets before
|
||||
> anything connects, each entry `name[:FORMAT]` with an unsuffixed name meaning
|
||||
> ICEBERG. The same list can come from `S3_TABLE_BUCKET`.
|
||||
>
|
||||
> Buckets created before this field existed carry no declaration and keep
|
||||
> accepting either format. See [[SeaweedFS Lance Catalog]].
|
||||
|
||||
|
||||
@@ -39,11 +39,33 @@ or `weed mini`, which starts the Lance namespace along with everything else.
|
||||
|
||||
### 2. Create a Lance table bucket
|
||||
|
||||
Two ways, depending on whether the cluster is already running.
|
||||
|
||||
**From the shell**, against a running cluster:
|
||||
|
||||
```bash
|
||||
weed shell
|
||||
> s3tables.bucket -create -name vectors -format LANCE -account 000000000000
|
||||
```
|
||||
|
||||
**At startup**, with `weed mini -tableBucket`, which pre-creates buckets before
|
||||
anything connects. Each entry is `name[:FORMAT]`, and an unsuffixed name means
|
||||
`ICEBERG`:
|
||||
|
||||
```bash
|
||||
weed mini -tableBucket=vectors:LANCE
|
||||
|
||||
# several at once, in either format
|
||||
weed mini -tableBucket=warehouse,vectors:LANCE
|
||||
```
|
||||
|
||||
The same list can come from the `S3_TABLE_BUCKET` environment variable, which is
|
||||
how the Docker images take it.
|
||||
|
||||
mini will not create a bucket in a format it does not serve — a `:LANCE` entry
|
||||
with the Lance namespace disabled is skipped with a warning rather than made
|
||||
unreachable — so keep `-s3.port.lance` enabled (it is, by default).
|
||||
|
||||
A bucket holds one format. Declaring `LANCE` means the catalog refuses an Iceberg table in it, and the reverse. Buckets created before formats were declared carry none and keep accepting either.
|
||||
|
||||
### 3. Use it
|
||||
|
||||
@@ -10,13 +10,15 @@ the integration suite in `test/s3tables/catalog_spark_lance/`.
|
||||
## Prerequisites
|
||||
|
||||
- SeaweedFS with the Lance Namespace enabled: `weed server -s3 -s3.port.lance=9101`
|
||||
- A table bucket declared `LANCE`:
|
||||
- A table bucket declared `LANCE`, made either from the shell:
|
||||
|
||||
```bash
|
||||
weed shell
|
||||
> s3tables.bucket -create -name vectors -format LANCE -account 000000000000
|
||||
```
|
||||
|
||||
or at startup, with `weed mini -tableBucket=vectors:LANCE`.
|
||||
|
||||
## Configuration
|
||||
|
||||
### PySpark
|
||||
|
||||
Reference in New Issue
Block a user