Commit Graph
31 Commits
Author SHA1 Message Date
Chris LuandDevin 80a26020d7 util: serialize all ViperProxy access so startup cannot hit concurrent map read/write (#11470)
* util: serialize every ViperProxy method; stop promoting unlocked viper calls

ViperProxy embedded *viper.Viper, so only the five declared methods took
the mutex while every promoted call — GetStringMap in backend.LoadConfiguration
was the reported crash — touched viper's maps unsynchronized. `weed server`
starts the volume server (SetDefault writer) and the master (GetStringMap
reader) back to back, and a race build reports the pair on a plain start.

The wrapped viper is now a named field: a method must be declared here to
exist on the proxy, so unsynchronized access fails at compile time rather
than at runtime. Every promoted use in the tree (GetStringMap, GetUint32,
GetFloat64, GetDuration, IsSet, AllKeys, Set) gets a locked wrapper;
NewViperProxy replaces struct literals for local vipers. GetStringMap
deep-copies its result — viper hands back the internal subtree, so
iterating it after the lock is released would race the next writer.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* util: take the shared lock while LoadConfiguration merges a config file

viper.MergeInConfig rewrites the same maps the proxy serializes; without
the lock a merge can race a concurrent SetDefault or reader exactly like
the reported startup crash.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* util: deep-copy slice elements in the GetStringMap snapshot

A slice of maps inside the returned subtree still shared the inner maps —
copy elements recursively so nothing the caller mutates is viper's
internal state.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* util: add the missing AutomaticEnv wrapper used by tests

sse_reader_test reaches it through GetViper(); without the wrapper the
call no longer exists once the viper field stopped being embedded.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* util: return a fresh slice from GetStringSlice

A stored []string comes back uncast from viper — the backing array is
shared internal state like the GetStringMap subtree, so copy it while
holding the lock.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-26 16:09:29 +08:00
chrislu 58acc14d2c avoid unnecessary fail fast
fix https://github.com/seaweedfs/seaweedfs/issues/7417
2025-10-31 12:49:04 -07:00
chrislu ec155022e7 "golang.org/x/exp/slices" => "slices" and go fmt 2024-12-19 19:25:06 -08:00
vadimartynov b796c21fa9 Added loadSecurityConfigOnce (#5792) 2024-07-16 09:15:55 -07:00
chrislu 26dbc6c905 move to https://github.com/seaweedfs/seaweedfs 2022-07-29 00:17:28 -07:00
chrislu 3da2b83b38 Added a "-conf_dir" option to customize *.toml configuration file directory.
fix https://github.com/chrislusf/seaweedfs/issues/2753
2022-03-19 00:22:47 -07:00
Chris Lu dce1f02c9e filer.backup: backup to local directory optionally is incremental
fixed one issue with https://github.com/chrislusf/seaweedfs/issues/2084
2021-05-25 17:19:20 -07:00
Chris Lu 52a8f1470e filer store: add mysql2 2021-01-19 17:21:50 -08:00
Chris Lu e2c7e3fe6d better locks 2021-01-13 13:20:33 -08:00
Chris Lu 819a85f59a avoid viper concurrent access 2021-01-13 13:14:52 -08:00
Chris Lu cfb9342a15 avoid concurrent map updates to viper 2021-01-12 02:28:13 -08:00
Chris Lu 4f31c1bb94 go fmt 2020-12-22 02:34:08 -08:00
Chris Lu 06bb7bf6c0 adding /usr/local/etc/seaweedfs for bsd style config search path
related to https://reviews.freebsd.org/D27391
2020-12-07 16:46:48 -08:00
Chris Lu 5b92dfe302 adjust log level 2020-10-23 23:58:54 -07:00
Chris Lu 11716fbf6f make reading error more obvious 2020-10-22 13:33:45 -07:00
wuyuxiang b556d3d035 Fix data race about config 2020-04-14 10:22:40 +08:00
Chris Lu d6412f27a0 logging 2020-03-22 18:32:56 -07:00
Chris Lu d335f04de6 support env variables to overwrite toml file 2020-01-29 09:09:55 -08:00
Chris Lu 27b94cb65b fix wrong url
fix https://github.com/chrislusf/seaweedfs/issues/1187
2020-01-28 00:49:47 -08:00
Chris Lu 573b74f24f add SetDefault function for Configuration 2019-12-20 20:56:14 -08:00
Chris Lu 0da7b894cc pass backend config from master to volume servers 2019-11-29 01:05:09 -08:00
Chris Lu de0c983808 refactoring scaffolding for master.toml 2019-11-14 22:20:38 -08:00
Lei Liu 46755ea1e1 fix master maintenance logic
Signed-off-by: Lei Liu <lei01.liu@horizon.ai>
2019-11-12 14:46:10 +08:00
stlpmo d07701fa75 delete unused function 2019-11-11 18:08:48 +08:00
stlpmo 62d393d6c9 ut pass 2019-11-11 10:52:21 +08:00
stlpmo 802a0eb3fe move from cmd-line to scaffold 2019-11-11 09:15:17 +08:00
Chris Lu ede876cfdb periodic scripts exeuction from leader master 2019-06-05 01:30:24 -07:00
Chris Lu f827ada811 merge notification config with filer.toml 2018-08-19 15:17:55 -07:00
Chris Lu 5ce6bbf076 directory structure change to work with glide
glide has its own requirements. My previous workaround caused me some
code checkin errors. Need to fix this.
2016-06-02 18:09:14 -07:00
Chris Lu 5071f528f6 testing compilation with remove package 2013-02-10 03:49:51 -08:00
Chris Lu ab6fb13ad7 avoid the "src" folder 2013-02-10 03:25:35 -08:00