Lines Matching defs:doclist
178763 ** A doclist (document list) holds a docid-sorted list of hits for a
178772 ** A doclist is stored like this:
178775 ** varint docid; (delta from previous doclist)
178813 ** a doclist consists of one or more document records.
178815 ** A bare doclist omits the position information, becoming an
178828 ** varint nDoclist; (length of term's associated doclist)
178829 ** char pDoclist[nDoclist]; (content of doclist)
178835 ** varint nDoclist; (length of term's associated doclist)
178836 ** char pDoclist[nDoclist]; (content of doclist)
178850 ** node (a leaf node with a single term and doclist). The goal of
178976 ** optimizations to things like doclist merging will swing the sweet
178985 ** write an empty doclist (varint(docid) varint(POS_END)), for updates
178986 ** we simply write the new doclist. Segment merges overwrite older
179583 /* True to disable the incremental doclist optimization. This is controled
179584 ** by special insert command 'test-no-incr-doclist'. */
179663 char *aAll; /* Array containing doclist (or NULL) */
179693 /* Cache of doclist for this phrase. */
179694 Fts3Doclist doclist;
179695 int bIncr; /* True if doclist is loaded incrementally */
179716 ** of this phrase query in FTS3 doclist format. As usual, the initial
179848 char *aDoclist; /* Pointer to doclist buffer */
180130 ** Read a single varint from the doclist at *pp and advance *pp to point
180142 ** varint that is part of a doclist (or position-list, or any other list
180146 ** Argument pStart points to the first byte of the doclist that the
180158 ** interested in. So, unless the doclist is corrupt, the 0x80 bit is
181724 ** a single document record of a doclist. So, in other words, this
181726 ** the doclist, or to the first byte past the end of the doclist.
181768 ** within a single document within a doclist.
181946 ** the part of a doclist that follows each document id. For example, if a row
182218 ** positions contained in either argument doclist). If the docids in the
182224 ** containing the output doclist and SQLITE_OK is returned. In this case
182225 ** *pnOut is set to the number of bytes in the output doclist.
182232 char *a1, int n1, /* First doclist */
182233 char *a2, int n2, /* Second doclist */
182234 char **paOut, int *pnOut /* OUT: Malloc'd doclist */
182323 ** doclist for which there is a position in the left-hand input doclist
182330 ** The right-hand input doclist is overwritten by this function.
182335 char *aLeft, int nLeft, /* Left doclist */
182336 char **paRight, int *pnRight /* IN/OUT: Right/output doclist */
182448 ** doclist stored in TermSelect.aaOutput[0]. If successful, delete all
182461 ** into a single doclist.
182496 ** Merge the doclist aDoclist/nDoclist into the TermSelect object passed
182500 ** This function is called with the doclist for each term that matches
182501 ** a queried prefix. It merges all these doclists into one, the doclist
182512 char *aDoclist, /* Pointer to doclist */
182516 /* If this is the first term selected, copy the doclist to the output
182521 ** to hold the current doclist AND'd with any other doclist. If the
182525 ** not true for order=DESC. For example, a doclist containing (1, -1)
182709 ** passed as the 4th argument also scan the doclist for term zTerm/nTerm.
182716 const char *zTerm, /* Term to scan doclist of */
182726 ** Open an Fts3MultiSegReader to scan the doclist for term zTerm/nTerm. Or,
182727 ** if isPrefix is true, to scan the doclist for all terms for which
182801 ** This function retrieves the doclist for the specified term (or term
182813 TermSelect tsc; /* Object for pair-wise doclist merging */
182853 ** This function counts the total number of docids in the doclist stored
182856 ** If the isPoslist argument is true, then it is assumed that the doclist
182858 ** that the doclist is simply a list of docids stored as delta encoded
183302 ** is the first in the doclist. In this case do not skip forward 2 bytes.
183304 ** is required for cases where the first byte of a doclist and the
183305 ** doclist is empty. For example, if the first docid is 10, a doclist
183802 ** doclist and then traversed.
183837 ** Arguments pList/nList contain the doclist for token iToken of phrase p.
183838 ** It is merged into the main doclist stored in p->doclist.aAll/nAll.
183850 char *pList, /* Pointer to doclist */
183857 sqlite3_free(p->doclist.aAll);
183858 p->doclist.aAll = 0;
183859 p->doclist.nAll = 0;
183863 p->doclist.aAll = pList;
183864 p->doclist.nAll = nList;
183867 else if( p->doclist.aAll==0 ){
183879 pLeft = p->doclist.aAll;
183880 nLeft = p->doclist.nAll;
183885 pRight = p->doclist.aAll;
183886 nRight = p->doclist.nAll;
183896 p->doclist.aAll = pRight;
183897 p->doclist.nAll = nRight;
183905 ** Load the doclist for phrase p into p->doclist.aAll/nAll. The loaded doclist
183942 ** means that the phrase does not appear in the current row, doclist.pList
183943 ** and doclist.nList are both zeroed.
183952 char *aFree = (pPhrase->doclist.bFreeList ? pPhrase->doclist.pList : 0);
183967 pPhrase->doclist.pList = 0;
183968 pPhrase->doclist.nList = 0;
183988 pPhrase->doclist.pList = 0;
183989 pPhrase->doclist.nList = 0;
184000 pPhrase->doclist.pList = aPoslist;
184001 pPhrase->doclist.nList = nPoslist;
184002 pPhrase->doclist.iDocid = pCsr->iPrevId;
184003 pPhrase->doclist.bFreeList = 1;
184012 p2 = pPhrase->doclist.pList;
184015 p1 = pPhrase->doclist.pList;
184026 pPhrase->doclist.pList = aOut;
184029 pPhrase->doclist.bFreeList = 1;
184030 pPhrase->doclist.nList = (int)(aOut - pPhrase->doclist.pList);
184033 pPhrase->doclist.pList = 0;
184034 pPhrase->doclist.nList = 0;
184040 if( pPhrase->doclist.pList!=aFree ) sqlite3_free(aFree);
184058 ** incremental loading strategy. Otherwise, the entire doclist is loaded into
184101 /* Load the full doclist for the phrase into memory. */
184116 ** The doclist may be sorted in ascending (parameter bDescIdx==0) or
184118 ** function iterates from the end of the doclist to the beginning.
184121 int bDescIdx, /* True if the doclist is desc */
184122 char *aDoclist, /* Pointer to entire doclist */
184174 ** Iterate forwards through a doclist.
184177 int bDescIdx, /* True if the doclist is desc */
184178 char *aDoclist, /* Pointer to entire doclist */
184229 /* We have already reached the end of this doclist. EOF. */
184281 TokenDoclist *p, /* OUT: Docid and doclist for new entry */
184289 fts3EvalDlPhraseNext(pTab, &pPhrase->doclist, pbEof);
184290 p->pList = pPhrase->doclist.pList;
184291 p->nList = pPhrase->doclist.nList;
184292 p->iDocid = pPhrase->doclist.iDocid;
184315 ** * features at least one token that uses an incremental doclist, and
184335 Fts3Doclist *pDL = &p->doclist;
184436 Fts3Doclist *pDL = &p->doclist;
184461 ** Fts3Phrase.doclist.aAll/nAll variables contain the entire doclist for
184464 ** may be loaded incrementally, meaning doclist.aAll/nAll is not available.
184511 int nOvfl; /* Number of overflow pages to load doclist */
184580 ** of each doclist, has not yet been determined. Read the required
184680 ** to retrieve the entire doclist for the token from the full-text index.
184688 ** After each token doclist is loaded, merge it with the others from the
184689 ** same phrase and count the number of documents that the merged doclist
184691 ** any phrase doclist for which 1 or more token doclists have been loaded.
184695 ** Then, for each token, defer it if loading the doclist would result in
184731 ** part of a multi-token phrase. Either way, the entire doclist will
184746 pTC->pPhrase->doclist.aAll, pTC->pPhrase->doclist.nAll
184820 if( pPhrase->doclist.bFreeList ){
184821 sqlite3_free(pPhrase->doclist.pList);
184823 pPhrase->doclist.pList = 0;
184824 pPhrase->doclist.nList = 0;
184825 pPhrase->doclist.bFreeList = 0;
184855 Fts3Phrase *pPhrase /* The phrase object to trim the doclist of */
184864 assert( pPhrase->doclist.pList );
184866 p2 = pOut = pPhrase->doclist.pList;
184871 nNew = (int)(pOut - pPhrase->doclist.pList) - 1;
184872 assert_fts3_nc( nNew<=pPhrase->doclist.nList && nNew>0 );
184873 if( nNew>=0 && nNew<=pPhrase->doclist.nList ){
184874 assert( pPhrase->doclist.pList[nNew]=='\0' );
184875 memset(&pPhrase->doclist.pList[nNew], 0, pPhrase->doclist.nList - nNew);
184876 pPhrase->doclist.nList = nNew;
184878 *paPoslist = pPhrase->doclist.pList;
184902 ** FTs3Expr.pPhrase->doclist.nList (length of pList in bytes)
184903 ** FTs3Expr.pPhrase->doclist.pList (pointer to position list)
184972 if( pRight->pPhrase->doclist.aAll ){
184973 Fts3Doclist *pDl = &pRight->pPhrase->doclist;
184979 if( pLeft->pPhrase && pLeft->pPhrase->doclist.aAll ){
184980 Fts3Doclist *pDl = &pLeft->pPhrase->doclist;
185047 pExpr->iDocid = pPhrase->doclist.iDocid;
185060 ** (Fts3Expr->pPhrase.doclist.pList/nList) for each phrase in the expression.
185102 assert( p->pRight->pPhrase->doclist.nList>0 );
185103 nTmp += p->pRight->pPhrase->doclist.nList;
185105 nTmp += p->pPhrase->doclist.nList;
185111 char *aPoslist = p->pPhrase->doclist.pList;
185120 aPoslist = pExpr->pRight->pPhrase->doclist.pList;
185167 /* If the NEAR expression does not match any rows, zero the doclist for
185216 pExpr->iDocid==pCsr->iPrevId && pExpr->pPhrase->doclist.pList
185223 bHit = (pPhrase->doclist.pList!=0);
185230 && pExpr->pPhrase->doclist.nList>0
185356 pPhrase->doclist.pNextDocid = 0;
185357 pPhrase->doclist.iDocid = 0;
185375 ** found in Fts3Expr.pPhrase->doclist.pList for each of the phrase
185381 if( pPhrase && pPhrase->doclist.pList ){
185383 char *p = pPhrase->doclist.pList;
185582 Fts3Expr *pExpr, /* Phrase to return doclist for */
185601 pIter = pPhrase->doclist.pList;
185613 ** pCsr->iPrevId may lie earlier in the doclist buffer. Or, if the
185625 ** an incremental phrase. Load the entire doclist for the phrase
185659 bEof = !pPh->doclist.nAll ||
185660 (pIter >= (pPh->doclist.aAll + pPh->doclist.nAll));
185663 bDescDoclist, pPh->doclist.aAll, pPh->doclist.nAll,
185668 bEof = !pPh->doclist.nAll || (pIter && pIter<=pPh->doclist.aAll);
185672 bDescDoclist, pPh->doclist.aAll, pPh->doclist.nAll,
185714 ** * the contents of pPhrase->doclist, and
185720 sqlite3_free(pPhrase->doclist.aAll);
185722 memset(&pPhrase->doclist, 0, sizeof(Fts3Doclist));
190049 char *aDoclist; /* Pointer to doclist of current entry */
190050 int nDoclist; /* Size of doclist in current entry */
190053 ** through the current doclist (aDoclist/nDoclist).
191311 /* Check that the doclist does not appear to extend past the end of the
191312 ** b-tree node. And that the final byte of the doclist is 0x00. If either
191325 ** Set the SegReader to point to the first docid in the doclist associated
191351 ** Advance the SegReader to point to the next docid in the doclist
191356 ** in the doclist entry (i.e. immediately past the docid varint).
191424 /* If there are no more entries in the doclist, set pOffsetList to
191712 ** a doclist for identical terms. Comparison is made as follows:
191714 ** 1) EOF (end of doclist in this case) is greater than not EOF.
192127 const char *aDoclist, /* Pointer to buffer containing doclist */
192128 int nDoclist /* Size of doclist in bytes */
192175 sqlite3Fts3VarintLen(nDoclist) + /* Size of doclist */
192211 sqlite3Fts3VarintLen(nDoclist) + /* Size of doclist */
192229 /* Append the prefix-compressed term and doclist to the buffer. */
192683 const char *zTerm, /* Term to iterate through a doclist for */
192731 ** then the entire doclist for the term is available in
192856 int nDoclist = 0; /* Size of doclist */
192857 sqlite3_int64 iPrev = 0; /* Previous docid stored in doclist */
192861 ** and a single term returned with the merged doclist.
192890 ** doclist. */
193626 const char *aDoclist; /* Pointer to doclist */
193627 int nDoclist; /* Size of doclist in bytes */
193823 ** Append a term and (optionally) doclist to the FTS segment node currently
193831 ** leaf node, the doclist is passed as aDoclist/nDoclist. For an internal
193859 /* Node must have already been started. There must be a doclist for a
193860 ** leaf node, and there must not be a doclist for an internal node. */
193892 ** Append the current term and doclist pointed to by cursor pCsr to the
193900 Fts3MultiSegReader *pCsr /* Cursor containing term and doclist */
193921 /* If the current block is not empty, and if adding this term/doclist
195331 }else if( nVal>21 && 0==sqlite3_strnicmp(zVal,"test-no-incr-doclist=",21) ){
195376 ** A deferred-doclist is like any other doclist with position information
196575 char *pIter = pPhrase->doclist.pList;
196646 ** file system. This is done because the full-text index doclist is required
196647 ** to calculate these values properly, and the full-text index doclist is
222545 void **ppObj, /* OUT: Pointer to doclist for pTerm */
222546 int *pnDoclist /* OUT: Size of doclist in bytes */
222557 const u8 **ppDoclist, /* OUT: pointer to doclist */
222558 int *pnDoclist /* OUT: size of doclist in bytes */
229467 ** to accumuluate "term -> doclist" content before it is flused to a level-0
229489 ** to the doclist data stored in the database. It is:
229929 ** Query the hash table for a doclist associated with term pTerm/nTerm.
229936 int *pnDoclist /* OUT: Size of doclist in bytes */
229988 const u8 **ppDoclist, /* OUT: pointer to doclist */
229989 int *pnDoclist /* OUT: size of doclist in bytes */
230035 ** (e.g. 1000 bytes). A single doclist may span multiple pages. Care is
230037 ** the entries in a doclist, or to seek to a specific entry within a
230038 ** doclist, without loading it into memory.
230040 ** * large doclists that span many pages have associated "doclist index"
230042 ** the doclist. This is used to speed up seek operations, and merges of
230109 ** Most of each segment leaf is taken up by term/doclist data. The
230110 ** general format of term/doclist, starting with the first term
230115 ** doclist: first doclist
230120 ** doclist: next doclist
230123 ** doclist format:
230169 ** The term/doclist format described above is accurate if the entire
230170 ** term/doclist data fits on a single leaf page. If this is not the case,
230184 ** 5. Segment doclist indexes:
230187 ** a single leaf record only. The format of each doclist index leaf page
230201 ** Internal doclist index nodes are:
230314 u8 *aEof; /* Pointer to 1 byte past end of doclist */
230366 u8 bFirstRowidInDoclist; /* True if next rowid is first in doclist */
230415 ** If set, set the iterator to point to EOF after the current doclist
230445 ** is the offset of the first rowid in the current doclist. */
230457 Fts5DlidxIter *pDlidx; /* If there is a doclist-index */
230533 ** of a doclist-index record.
230536 ** Record containing the doclist-index data.
231336 ** doclist-index page is reached, return non-zero.
231397 ** points to the first rowid in the doclist-index.
231400 ** pointer to doclist-index record,
231403 ** doclist is associated with (the one featuring the term).
231518 ** Free a doclist-index iterator object allocated by fts5DlidxIterInit().
232160 ** Iterator pIter currently points to the first rowid in a doclist. This
232162 ** the doclist.
232195 /* The last rowid in the doclist may not be on the current page. Search
232215 /* If pLast is NULL at this point, then the last rowid for this doclist
232248 ** Iterator pIter currently points to the first rowid of a doclist.
232249 ** There is a doclist-index associated with the final term on the current
232251 ** doclist-index from disk and initialize an iterator at (pIter->pDlidx).
232261 /* Check if the current doclist ends on this page. If it does, return
232262 ** early without loading the doclist-index (as it belongs to a different
232441 int bDlidx = 0; /* True if there is a doclist-index */
233554 ** Create an Fts5Iter that iterates through the doclist provided
233780 ** If the current doclist-index accumulating in pWriter->aDlidx[] is large
233788 ** to the database, also write the doclist-index to disk. */
233798 ** This function is called whenever processing of the doclist for the
233801 ** The doclist-index for that term is currently stored in-memory within the
233857 /* If there were no rowids on the leaf page either and the doclist-index
233881 ** doclist-index.
233896 /* The current doclist-index page is full. Write it to disk and push
233898 ** doclist-index leaf page) up into the next level of the b-tree
234084 ** buffer, in case a doclist-index is required. */
234609 ** term/doclist currently stored within the hash table. */
234615 const u8 *pDoclist; /* Pointer to doclist for this term */
234616 int nDoclist; /* Size of doclist in bytes */
234625 /* The entire doclist will fit on the current leaf. */
234632 /* The entire doclist will not fit on this leaf. The following
234634 ** doclist. */
234930 ** Append a doclist to buffer pBuf.
235059 ** doclist in buffer p1.
235078 /* Initialize a doclist-iterator for each input buffer. Arrange them in
235242 Fts5Buffer doclist;
235245 memset(&doclist, 0, sizeof(doclist));
235259 xAppend(p, (u64)p1->base.iRowid-(u64)iLastRowid, p1, &doclist);
235285 if( p1->base.iRowid<=iLastRowid && doclist.n>0 ){
235286 for(i=0; p->rc==SQLITE_OK && doclist.n; i++){
235292 fts5BufferSwap(&doclist, &aBuf[iStore]);
235293 fts5BufferZero(&doclist);
235298 xMerge(p, &doclist, nMerge, &aBuf[i1]);
235307 xAppend(p, (u64)p1->base.iRowid-(u64)iLastRowid, p1, &doclist);
235315 xMerge(p, &doclist, nMerge, &aBuf[i]);
235323 pData = fts5IdxMalloc(p, sizeof(Fts5Data)+doclist.n+FTS5_DATA_ZERO_PADDING);
235326 pData->nn = pData->szLeaf = doclist.n;
235327 if( doclist.n ) memcpy(pData->p, doclist.p, doclist.n);
235330 fts5BufferFree(&doclist);
235820 ** visited regardless of whether the doclist-index identified by parameters
235826 int iLeaf /* Load doclist-index for this leaf */
236156 /* If there is a doclist-index, check that it looks right. */
236158 Fts5DlidxIter *pDlidx = 0; /* For iterating through doclist index */
236203 /* TODO: Check there is no doclist index */
236464 ** The start of buffer (a/n) contains the start of a doclist. The doclist
236466 ** representation of the part of the doclist that is present to buffer
236502 ** Buffer (pData/nData) contains a doclist in the format used by detail=none
236629 /* Figure out where the doclist for this term ends */
236686 /* Decode any more doclist data that appears on the page before the