Lines Matching defs:data
12 the gzip file, only compressing when the amount of uncompressed data has
18 valid state. The last data to be appended or compressed is saved in an
22 gzlog maintains another auxiliary file with the last 32K of data from the
24 data. This minimizes the impact to the compression ratio of appending.
32 foo.add -- last message to append or last data to compress
33 foo.dict -- dictionary of the last 32K of data for next compression
40 - compressed data ending initially with empty stored block
41 - uncompressed data filling out originally empty stored block and
46 When appending data, the information in the first three items above plus the
49 stored block and determine where to append the data in the foo.add file, as
50 well as the crc and length of the gzip data before the append operation.
54 operation is interrupted, the data to add will still be there. If due to
57 still be restored, but without the appended data.
59 When compressing data, the information in the first two items above plus the
62 compressed data, and contains both the crc and length of just the compressed
63 data and of the complete set of data including the contents of the foo.add
67 of an interruption. If in the unlikely event the foo.add file with the data
69 just the previous compressed data will be reconstructed. In this case, all
70 of the data that was to be compressed is lost (approximately one megabyte).
116 for the last stored block of the uncompressed data in the gzip file.
123 - Compressed data crc and length. This is the crc and length of the data
125 only in the event that the foo.add file containing the data to compress is
127 - Total data crc and length. This is the crc and length of all of the data
141 also allows for new compressed data to be appended to the old compressed
142 data in the compress operation, overwriting the previous first stored
143 block, or for the compressed data to be terminated and a valid gzip file
145 interrupted and the data to compress in the foo.add file was deleted.
159 - Perform the append procedure with the provided data.
160 - If the uncompressed data in the foo.gz file is 1MB or more, apply the
171 - Append the provided data to the last stored block, creating new stored
174 - Update the crc and length with the new data, and write the gzip trailer.
179 completed operation, it is truncated when writing new data to it.
183 - Read all of the uncompressed data in the stored blocks in foo.gz and write
184 it to foo.add. Also write foo.temp with the last 32K of that data to
187 * If there is no data provided to compress (due to a missing foo.add file
189 only the previous compressed data and proceed to the step after the next
191 - Compress the data with the dictionary in foo.dict, and write to the
201 - Append the crc and length of the data in the gzip file (previously
211 - If not a replace recovery, read in the foo.add file, and provide that data
213 a zero data length to the recovery. In that case, the append recovery
214 restores the foo.gz to the previous compressed + uncompressed data state.
216 restored to the previous compressed-only data state.
298 ulong ccrc; /* crc of compressed data */
299 ulong clen; /* length (modulo 2^32) of compressed data */
300 ulong tcrc; /* crc of total data */
301 ulong tlen; /* length (modulo 2^32) of total data */
322 0, 0, 0, 0, 0, 0, 0, 0, /* compressed data crc and length */
323 0, 0, 0, 0, 0, 0, 0, 0, /* total data crc and length */
324 0, 0, /* final stored block data length */
330 /* initial gzip data and trailer */
460 the file pointer after the end of the last stored block data. Return -1 if
488 pointer to after the last stored block data */
496 /* Append len bytes from data to the locked and open log file. len may be zero
498 of the foo.gz file is restored. The data is appended uncompressed in
501 local int log_append(struct log *log, unsigned char *data, size_t len)
521 if (write(log->fd, data, put) != put)
524 log->tcrc = crc32(log->tcrc, data, put);
527 data += put;
598 /* Compress the len bytes at data and append the compressed data to the
599 foo.gz deflate data immediately after the previous compressed data. This
600 overwrites the previous uncompressed data, which was stored in foo.add
601 and is the data provided in data[0..len-1]. If this operation is
603 file read in again. If there is no data to compress (len == 0), then we
604 simply terminate the foo.gz file after the previously compressed data,
608 local int log_compress(struct log *log, unsigned char *data, size_t len)
617 /* compress and append compressed data */
627 /* read in dictionary (last 32K of data that was compressed) */
653 strm.next_in = data;
694 /* no data to compress -- fix up existing gzip stream */
741 unsigned char *data = NULL;
754 (data = malloc(st.st_size)) == NULL) {
759 free(data);
763 ret = (size_t)read(fd, data, len) != len;
766 free(data);
779 ret = log_append(log, data, len);
782 ret = log_compress(log, data, len);
792 if (data != NULL)
793 free(data);
915 unsigned char *data, buf[5];
928 /* create space for uncompressed data */
931 if ((data = malloc(len)) == NULL)
936 /* read in the uncompressed data */
945 read(log->fd, (char *)data + next, block) != block)
953 /* write the uncompressed data to the .add file */
958 ret = (size_t)write(fd, data, len) != len;
969 ret = (size_t)write(fd, (char *)data + len - next, next) != next;
974 /* roll back to compressed data, mark the compress in progress */
981 /* compress and append the data (clears mark) */
982 ret = log_compress(log, data, len);
983 free(data);
988 free(data);
997 int gzlog_write(gzlog *logd, void *data, size_t len)
1005 if (data == NULL || len <= 0)
1019 ret = (size_t)write(fd, data, len) != len;
1029 /* append data (clears mark) */
1030 if (log_append(log, data, len))