Commit Graph
190 Commits
Author SHA1 Message Date
chrislu d6a066495b adjust logs 2024-03-25 08:15:47 -07:00
chrislu 1b1b96ea1e close local partition if no more publishers and subscribers 2024-03-24 21:10:02 -07:00
chrislu 4f5c4c3388 refactor 2024-03-24 13:07:51 -07:00
chrislu d558a58946 refactor 2024-03-24 13:04:59 -07:00
chrislu 205d2285f3 setup follower by publisher
* the subscriber would getOrGen a local partition and wait
* the publisher would getOrGen a local partition. If localPartition follower is not setup, and init message has follower info, it would create followers based on init.Messages.
2024-03-24 12:57:09 -07:00
chrislu 7003ce7425 publisher is able to fully send data and receive acks
still needs to close the pipes cleanly
2024-03-20 12:25:40 -07:00
chrislu 516cba7053 remove FollowInMemoryMessages 2024-03-17 22:27:13 -07:00
chrislu a375b2815e Merge branch 'master' into mq-subscribe 2024-03-17 11:46:55 -07:00
chrislu a8b73296d7 fix test3 2024-03-16 23:53:46 -07:00
chrislu 8e5068fd2f notify 2024-03-16 23:16:33 -07:00
chrislu 44ab42d03c unnecessary 2024-03-16 17:11:42 -07:00
chrislu 3f2dd8cc3e add follower id info 2024-03-16 17:11:18 -07:00
chrislu 529fa8c4c8 go fmt 2024-03-16 10:51:47 -07:00
chrislu 1b5c547cfe logs 2024-03-15 21:08:57 -07:00
chrislu 55714f54ab local partition is generated or not 2024-03-15 21:08:42 -07:00
chrislu 549455c30d streaming mode 2024-03-15 21:05:24 -07:00
chrislu f653838f34 follower receive a hello message before actual data
the leader can ensure the follower has connected
2024-03-15 20:36:01 -07:00
chrislu 6f75a0af55 ask follower to follow 2024-03-10 14:34:28 -07:00
chrislu e7823ee967 retry connecting to broker leader 2024-03-09 12:56:49 -08:00
chrislu 28f8f33d6a include key in LogEntry 2024-03-07 10:53:30 -08:00
chrislu 34f2b600ac each log function adds a "done" return parameter 2024-03-07 10:50:09 -08:00
chrislu 62397f2371 adjust error 2024-03-07 10:47:50 -08:00
chrislu 7598922b41 assign followers 2024-02-29 14:51:06 -08:00
chrislu 1b4484bf0a go fmt 2024-02-29 09:38:52 -08:00
Chris Lu 16b4f9d9ab subscriber find broker leader first 2024-02-05 23:14:25 -08:00
Chris Lu 656b78d1af proxy to broker leader 2024-02-05 18:00:50 -08:00
Chris Lu 4dc6681833 proxy requests to lock owner 2024-02-05 16:46:40 -08:00
Chris Lu f9131de43f fix lock TTL 2024-02-04 12:47:21 -08:00
Chris Lu 0a12301b3d avoid too large expiration time 2024-02-04 09:20:21 -08:00
chrislu d41792461c lock returns host and owner 2024-02-02 15:54:57 -08:00
chrislu d30150dde1 use a short-lived lock 2024-02-01 23:01:44 -08:00
chrislu ac50d8a822 re-lock if the lock owner is not found 2024-01-29 22:46:23 -08:00
chrislu 0b2e5ddc7c wait 3 seconds before shutting down publish client, to wait for all messages to be received 2024-01-28 22:04:42 -08:00
chrislu a507069d19 lock for creating local partition 2024-01-28 15:55:26 -08:00
chrislu 9e6ea80dff read existing topic conf 2024-01-28 14:27:52 -08:00
chrislu d40225ecd2 return the lookup results 2024-01-28 12:05:44 -08:00
chrislu 841fafd0a8 publish to input buffer
currently, the input buffer may not exist when start to publish
2024-01-27 23:43:22 -08:00
chrislu d268fbe18a when configure, cancel existing assignments 2024-01-22 11:57:17 -08:00
chrislu 3ff6b31d94 sub broker wait for partition 2024-01-22 11:56:55 -08:00
chrislu 2beaa2d0b3 pub/sub brokers check filer for assigned partitions 2024-01-22 11:44:21 -08:00
chrislu c3f8530f97 refactor 2024-01-22 11:06:53 -08:00
chrislu 5f35857cd0 no need to assign partitions explicitly 2024-01-22 11:04:47 -08:00
chrislu 428fea45f3 updated and added assignments 2024-01-22 11:04:24 -08:00
chrislu e8b05ecc91 add/remove assigned partitions 2024-01-22 10:47:39 -08:00
chrislu 861ad732ca refactor 2024-01-22 09:00:22 -08:00
chrislu 47924afa1c refactor 2024-01-22 08:52:44 -08:00
chrislu 581cf6644c lookup reads from filer instead of local memory 2024-01-22 00:52:55 -08:00
chrislu 7121ae9617 separate out read topic conf and ensure topic active assignments 2024-01-22 00:51:31 -08:00
chrislu c77d35313e pub/sub broker only check local assigned partitions 2024-01-22 00:49:57 -08:00
chrislu b0a2e9aea3 fix assignments if brokers changed 2024-01-21 01:27:22 -08:00