Commit Graph
8 Commits
Author SHA1 Message Date
Eugeniy E. Mikhailov 3003c9e17e added backward compatible CRC check and extra explanatory comments (#6233) 2024-11-12 13:56:14 -08:00
Eugeniy E. Mikhailov 0da9e40abb Feature: compress before encryption (#6013) 2024-09-15 08:47:53 -07:00
Eugeniy E. Mikhailov dab0bb8097 Feature limit caching to prescribed number of bytes per file (#6009)
* feature: we can check if a fileId is already in the cache

We using this to protect cache from adding the same needle to
the cache over and over.

* fuse mount: Do not start dowloader if needle is already in the cache

* added maxFilePartSizeInCache property to ChunkCache

If file very large only first maxFilePartSizeInCache bytes
are going to be put to the cache (subject to the needle size
constrains).

* feature: for large files put in cache no more than prescribed number of bytes

Before this patch only the first needle of a large file was intended for
caching. This patch uses maximum prescribed amount of bytes to be put in
cache. This allows to bypass default 2MB maximum for a file part stored
in the cache.

* added dummy mock methods to satisfy interfaces of ChunkCache
2024-09-11 21:09:20 -07:00
Eugeniy E. Mikhailov c04edeed68 bug fix in the data received from cache processing (#6002)
The patch addresses #3745.

The cache should return the exact amount of data requested by the buffer.
By construction of the cache it is always all requested data range
or we have error happening.

The old use of minsize miscalculate the requested data size,
if non zero offset is requested.
2024-09-10 13:30:18 -07:00
Eugeniy E. Mikhailov d6b0e0ff1c Remove "Content-Length" header if http.Error is issued (#5981) 2024-09-05 17:31:28 -07:00
Eugeniy E. Mikhailov bc01f09e37 Do CRC check if the buffer contains the full needle data before it is sent (#5980) 2024-09-05 17:14:55 -07:00
Eugeniy E. Mikhailov 282867a71e Show volume server which failed to verify a chunk (#5958) 2024-09-01 21:13:22 -07:00
Eugeniy E. Mikhailov 220061573d update metric for ErrorCRC if it happens during volume readNeedleDataInto (#5957) 2024-09-01 20:30:09 -07:00