Lines Matching refs:buddy
37 * - mb_mark_used() may allocate chunk right after splitting buddy
91 * prealloc then we look at the buddy cache. The buddy cache is represented
93 * mapped to the buddy and bitmap information regarding different
94 * groups. The buddy information is attached to buddy cache inode so that
97 * block bitmap and buddy information. The information are stored in the
101 * [ group 0 bitmap][ group 0 buddy] [group 1][ group 1]...
104 * one block each for bitmap and buddy information. So for each group we
109 * The buddy cache inode is not stored on disk. The inode is thrown
112 * We look for count number of blocks in the buddy cache. If we were able
116 * Before allocating blocks via buddy cache we normalize the request
131 * The regular allocator (using the buddy cache) supports a few tunables.
137 * The regular allocator uses buddy scan only if the request len is power of
154 * request we will hit the buddy cache which will result in this prealloc
162 * - in-core buddy (actually includes buddy and bitmap)
182 * in-core buddy = on-disk bitmap + preallocation descriptors
193 * blocks: how many blocks marked used/free in on-disk bitmap, buddy and PA.
196 * - init buddy: buddy = on-disk + PAs
197 * - new PA: buddy += N; PA = N
199 * - discard inode PA buddy -= on-disk - PA; PA = 0
201 * - discard locality group PA buddy -= PA; PA = 0
202 * note: 'buddy -= on-disk - PA' is used to show that on-disk bitmap
210 * 1) if buddy is referenced, it's already initialized
211 * 2) while block is used in buddy and the buddy is referenced,
215 * on-disk bitmap if buddy has same bit set or/and PA covers corresponded
219 * - init buddy vs.
221 * blocks for PA are allocated in the buddy, buddy must be referenced
222 * until PA is linked to allocation group to avoid concurrent buddy init
227 * the simplest way would be to have buddy initialized by the discard
231 * the simplest way would be to have buddy initialized by the discard
257 * - buddy init as sum of on-disk bitmap and PAs is done atomically
259 * a special case when we've used PA to emptiness. no need to modify buddy
620 void *buddy;
627 buddy = mb_find_buddy(e4b, order, &max);
628 MB_CHECK_ASSERT(buddy);
631 MB_CHECK_ASSERT(buddy != buddy2);
637 if (mb_test_bit(i, buddy)) {
665 buddy = mb_find_buddy(e4b, 0, &max);
667 if (!mb_test_bit(i, buddy)) {
697 MB_CHECK_ASSERT(mb_test_bit(k + i, buddy));
715 void *buddy, ext4_grpblk_t first, ext4_grpblk_t len,
743 buddy + sbi->s_mb_offsets[min]);
773 void *buddy, void *bitmap, ext4_group_t group,
785 /* initialize buddy from bitmap which is aggregation
796 ext4_mb_mark_free_simple(sb, buddy, first, len, grp);
830 void *buddy;
832 while ((buddy = mb_find_buddy(e4b, order++, &count)))
833 ext4_set_bits(buddy, 0, count);
844 /* The buddy information is attached the buddy cache inode
847 * block bitmap and buddy information. The information are
851 * [ group 0 bitmap][ group 0 buddy] [group 1][ group 1]...
854 * one block each for bitmap and buddy information.
952 /* skip initialized uptodate buddy */
970 * We place the buddy block and bitmap block
974 /* this is block of buddy */
976 mb_debug(sb, "put buddy for group %u in page %lu/%x\n",
992 /* init the buddy */
1013 /* set incore so that the buddy information can be
1032 * Lock the buddy and bitmap pages. This make sure other parallel init_group
1033 * on the same buddy page doesn't happen whild holding the buddy page lock.
1034 * Return locked buddy and bitmap pages on e4b struct. If buddy and bitmap
1050 * the buddy cache inode stores the block bitmap
1051 * and buddy information in consecutive blocks.
1065 /* buddy and bitmap are on the same page */
1112 * This ensures that we don't reinit the buddy cache
1114 * allocating. If we are looking at the buddy cache we would
1116 * would have pinned buddy page to page cache.
1140 * If both the bitmap and buddy are in
1142 * init the buddy
1147 /* init buddy cache */
1206 * the buddy cache inode stores the block bitmap
1207 * and buddy information in consecutive blocks.
1333 /* this block is part of buddy of order 'order' */
1424 void *buddy = mb_find_buddy(e4b, order, &max);
1426 while (buddy) {
1435 * single bit of highest order buddy.
1459 e4b->bd_info->bb_counters[order] += mb_buddy_adjust_border(&first, buddy, -1);
1461 e4b->bd_info->bb_counters[order] += mb_buddy_adjust_border(&last, buddy, 1);
1467 mb_clear_bits(buddy, first, last - first + 1);
1473 buddy = buddy2;
1538 /* buddy[0] == bd_bitmap is a special case, so handle
1566 void *buddy;
1571 buddy = mb_find_buddy(e4b, 0, &max);
1572 BUG_ON(buddy == NULL);
1574 if (mb_test_bit(block, buddy)) {
1635 void *buddy;
1658 /* let's maintain buddy itself */
1665 buddy = mb_find_buddy(e4b, ord, &max);
1667 mb_set_bit(start >> ord, buddy);
1679 /* we have to split large buddy */
1681 buddy = mb_find_buddy(e4b, ord, &max);
1682 mb_set_bit(start >> ord, buddy);
1687 buddy = mb_find_buddy(e4b, ord, &max);
1688 mb_clear_bit(cur, buddy);
1689 mb_clear_bit(cur + 1, buddy);
1960 * The routine scans buddy structures (not bitmap!) from given order
1969 void *buddy;
1979 buddy = mb_find_buddy(e4b, i, &max);
1980 BUG_ON(buddy == NULL);
1982 k = mb_find_next_zero_bit(buddy, max, 0);
2117 * This is also called BEFORE we load the buddy bitmap.
2216 * good chunks almost for free. If buddy data is not
2288 * need to make sure that the buddy bitmap in the page cache has been
2293 * TODO: We should actually kick off the buddy bitmap setup in a work
2353 * try exact allocation using buddy.
2358 * We search using buddy data only if the order of the request
2362 * requests upto maximum buddy size we have constructed.
2431 /* This now checks without needing the buddy page */
2695 ext4_msg(sb, KERN_ERR, "can't allocate buddy meta group");
2735 "for a buddy group");
2748 ext4_msg(sb, KERN_ERR, "can't allocate buddy mem");
2810 * use EXT4_BAD_INO for the buddy cache inode number. This inode is
3022 /* init file for buddy data */
3155 /* we expect to find existing buddy because it's pinned */
3163 /* there are blocks to put in buddy to make them really free */
3979 * buddy must be generated from this bitmap
4004 * used in in-core bitmap. buddy must be generated from this bitmap
4027 * allocation in buddy when concurrent ext4_mb_put_pa()
4114 * P1 (buddy init) P2 (regular allocation)
4116 * copy on-disk bitmap to buddy
4119 * mark all PAs in buddy
4121 * thus, P1 initializes buddy with B available. to prevent this
4303 * in-core bitmap and buddy.
4424 ext4_warning(sb, "Error %d loading buddy information for %u",
4581 ext4_error_err(sb, -err, "Error %d loading buddy information for %u",
4879 ext4_error_err(sb, -err, "Error %d loading buddy information for %u",
5282 protect buddy cache from being freed,
5749 * This marks the blocks as free in the bitmap and buddy.
5883 * @e4b: ext4 buddy for the group
5886 * one will allocate those blocks, mark it as used in buddy bitmap. This must
6001 * ext4_trim_all_free walks through group's buddy bitmap searching for free
6007 * extents. When the free extent is found, mark it as used in group buddy
6009 * the group buddy bitmap. This is done until whole group is scanned.
6023 ext4_warning(sb, "Error %d loading buddy information for %u",