Lines Matching defs:group
320 /* runqueue "owned" by this group */
437 /* Do the two (enqueued) entities belong to the same group ? */
505 /* runqueue "owned" by this group */
801 * nothing has been attached to the task group yet.
1321 static inline unsigned long group_faults_cpu(struct numa_group *group, int nid)
1323 return group->faults_cpu[task_faults_idx(NUMA_MEM, nid, 0)] + group->faults_cpu[task_faults_idx(NUMA_MEM, nid, 1)];
1354 * considered part of a numa group's pseudo-interleaving set. Migrations
1396 * of nodes, and move tasks towards the group with the most
1399 * of each group. Skip other nodes.
1433 * task group, on a particular numa node. The group weight is given a
1434 * larger multiplier, in order to group tasks together that are almost
1858 * If dst and source tasks are in the same NUMA group, or not
1859 * in any group then look only at task weights.
1866 * tasks within a group over tiny differences.
1873 * Compare the group weights. If a task is all by itself
1874 * (not part of a group), use the task weight instead.
2104 * multiple NUMA nodes; in order to better consolidate the group,
2396 * inside the highest scoring group of nodes. The nodemask tricks
2416 /* Sum group's NUMA faults; includes a==b case. */
2426 /* Remember the top group. */
2432 * just one node left in each "group", the
2472 /* If the task is part of a group prevent parallel updates to group stats */
2501 * Normalize the faults_from, so all tasks in a group
2518 * safe because we can only change our own group
2621 * Only join the other group if its bigger; if we're the bigger group,
3249 * That is, the weight of a group entity, is the proportional share of the
3250 * group weight based on the group runqueue weights. That is:
3276 * conditions. In specific, the case where the group was idle and we start the
3340 * of a group with small tg->shares value. It is a floor value which is
3342 * the group on a CPU.
3344 * E.g. on 64-bit for a group with tg->shares of scale_load(15)=15*1024
3357 * Recomputes the group entity based on the current state of its group
3498 * that for each group
3507 * the group entity and group rq do not have their PELT windows aligned).
3933 * track group sched_entity load average for task_h_load calc in migration
4446 * its group cfs_rq
4545 * - For group entity, update its weight to reflect the new share
4546 * of its group cfs_rq.
4821 * default period for cfs group bandwidth.
4935 * Ensure that neither of the group entities corresponding to src_cpu or
4936 * dest_cpu are members of a throttled hierarchy when performing group
4972 /* group is entering throttled state, stop time */
5382 * When a group wakes up we want to make sure that its quota is not already
5392 /* an active group must be handled by the update_curr()->put() path */
5397 /* ensure the group is not already throttled */
5561 * The race is harmless, since modifying bandwidth settings of unhooked group
6216 * find_idlest_group_cpu - find the idlest CPU among the CPUs in the group.
6218 static int find_idlest_group_cpu(struct sched_group *group, struct task_struct *p, int this_cpu)
6228 if (group->group_weight == 1) {
6229 return cpumask_first(sched_group_span(group));
6233 for_each_cpu_and(i, sched_group_span(group), p->cpus_ptr)
6293 struct sched_group *group;
6302 group = find_idlest_group(sd, p, cpu);
6303 if (!group) {
6308 new_cpu = find_idlest_group_cpu(group, p, cpu);
6703 * necessary. Without utilization capping, a group could be seen as overloaded
7109 * Balances load by selecting the idlest CPU in the idlest group, or under
7401 * group (e.g. via set_curr_task), since update_curr() (in the
7743 * `- size of each group
7809 * 'group_type' describes the group of CPUs at the moment of load balancing.
7811 * The enum is ordered by pulling priority, with the group with lowest priority
7813 * group. See update_sd_pick_busiest().
7816 /* The group has spare capacity that can be used to run more tasks. */
7819 * The group is fully used and the tasks don't compete for more CPU
8497 * group is a fraction of its parents load.
8585 unsigned long avg_load; /* Avg load across the CPUs of the group */
8586 unsigned long group_load; /* Total load over the CPUs of the group */
8588 unsigned long group_util; /* Total utilization over the CPUs of the group */
8589 unsigned long group_runnable; /* Total runnable time over the CPUs of the group */
8590 unsigned int sum_nr_running; /* Nr of tasks running in the group */
8591 unsigned int sum_h_nr_running; /* Nr of CFS tasks running in the group */
8608 struct sched_group *busiest; /* Busiest group in this sd */
8609 struct sched_group *local; /* Local group in this sd */
8615 struct sg_lb_stats busiest_stat; /* Statistics of the busiest group */
8616 struct sg_lb_stats local_stat; /* Statistics of the local group */
8625 * busiest_stat::idle_cpus to the worst busiest group because
8693 struct sched_group *group, *sdg = sd->groups;
8713 * span the current group.
8731 * span the current group.
8734 group = child->groups;
8736 struct sched_group_capacity *sgc = group->sgc;
8737 __maybe_unused cpumask_t *cpus = sched_group_span(group);
8744 group = group->next;
8745 } while (group != child->groups);
8778 * cpumask covering 1 CPU of the first group and 3 CPUs of the second group.
8784 * If we were to balance group-wise we'd place two tasks in the first group and
8785 * two tasks in the second group. Clearly this is undesired as it will overload
8786 * cpu 3 and leave one of the CPUs in the second group unused.
8788 * The current solution to this issue is detecting the skew in the first group
8792 * When this is so detected; this group becomes a candidate for busiest; see
8795 * to create an effective group imbalance.
8798 * group imbalance and decide the groups need to be balanced again. A most
8802 static inline int sg_imbalanced(struct sched_group *group)
8804 return group->sgc->imbalance;
8808 * group_has_capacity returns true if the group has spare capacity that could
8810 * We consider that a group has spare capacity if the * number of task is
8837 * group_is_overloaded returns true if the group has more tasks than it can
8839 * group_is_overloaded is not equals to !group_has_capacity because a group
8879 static inline enum group_type group_classify(unsigned int imbalance_pct, struct sched_group *group,
8886 if (sg_imbalanced(group)) {
8933 * @group: sched_group whose statistics are to be updated.
8934 * @sgs: variable to hold the statistics for this group.
8937 static inline void update_sg_lb_stats(struct lb_env *env, struct sched_group *group, struct sg_lb_stats *sgs,
8944 local_group = cpumask_test_cpu(env->dst_cpu, sched_group_span(group));
8946 for_each_cpu_and(i, sched_group_span(group), env->cpus)
8999 if (!group->group_weight) {
9003 sgs->group_weight = group->group_weight;
9007 /* Check if dst CPU is idle and preferred to this group */
9009 sched_asym_prefer(env->dst_cpu, group->asym_prefer_cpu)) {
9013 sgs->group_capacity = group->sgc->capacity;
9015 sgs->group_weight = group->group_weight;
9017 sgs->group_type = group_classify(env->sd->imbalance_pct, group, sgs);
9019 /* Computing avg_load makes sense only when group is overloaded */
9026 * update_sd_pick_busiest - return 1 on busiest group
9032 * Determine if @sg is a busier group than the previously selected
9033 * busiest group.
9035 * Return: %true if @sg is a busier group than the previously selected
9036 * busiest group. %false otherwise.
9051 * CPUs in the group should either be possible to resolve
9068 * The candidate and the current busiest group are the same type of
9069 * group. Let check which one is the busiest according to the type.
9074 /* Select the overloaded group with highest avg_load. */
9082 * Select the 1st imbalanced group as we don't have any way to
9106 * Select the fully busy group with highest avg_load. In
9108 * group because tasks have all compute capacity that they need
9122 * Select not overloaded group with lowest number of idle cpus
9125 * that the group has less spare capacity but finally more idle
9237 * @sd: The sched_domain level to look for idlest group.
9238 * @group: sched_group whose statistics are to be updated.
9239 * @sgs: variable to hold the statistics for this group.
9240 * @p: The task for which we look for the idlest group/CPU.
9242 static inline void update_sg_wakeup_stats(struct sched_domain *sd, struct sched_group *group, struct sg_lb_stats *sgs,
9249 for_each_cpu(i, sched_group_span(group))
9271 /* Check if task fits in the group */
9272 if (sd->flags & SD_ASYM_CPUCAPACITY && !task_fits_capacity(p, group->sgc->max_capacity)) {
9276 sgs->group_capacity = group->sgc->capacity;
9278 sgs->group_weight = group->group_weight;
9280 sgs->group_type = group_classify(sd->imbalance_pct, group, sgs);
9283 * Computing avg_load makes sense only when group is fully busy or
9291 static bool update_pick_idlest(struct sched_group *idlest, struct sg_lb_stats *idlest_sgs, struct sched_group *group,
9303 * The candidate and the current idlest group are the same type of
9304 * group. Let check which one is the idlest according to the type.
9310 /* Select the group with lowest avg_load. */
9322 /* Select group with the highest max capacity */
9323 if (idlest->sgc->max_capacity >= group->sgc->max_capacity) {
9329 /* Select group with most idle CPUs */
9334 /* Select group with lowest group_util */
9346 * find_idlest_group() finds and returns the least busy CPU group within the
9353 struct sched_group *idlest = NULL, *local = NULL, *group = sd->groups;
9372 /* Skip over this group if it has no CPUs allowed */
9374 if (!cpumask_intersects(sched_group_span(group), &allowed_cpus))
9376 if (!cpumask_intersects(sched_group_span(group), p->cpus_ptr))
9380 local_group = cpumask_test_cpu(this_cpu, sched_group_span(group));
9383 local = group;
9388 update_sg_wakeup_stats(sd, group, sgs, p);
9390 if (!local_group && update_pick_idlest(idlest, &idlest_sgs, group, sgs)) {
9391 idlest = group;
9394 } while (group = group->next, group != sd->groups);
9396 /* There is no idlest group to push tasks to */
9401 /* The local group has been skipped because of CPU affinity */
9407 * If the local group is idler than the selected idlest group
9415 * If the local group is busier than the selected idlest group
9439 * If the local group is less loaded than the selected
9440 * idlest group don't try and push any tasks.
9457 /* Select group with the highest max capacity */
9492 * Select group with highest number of idle CPUs. We could also
9494 * up that the group has less spare capacity but finally more
9637 * In the group_imb case we cannot rely on group-wide averages
9648 * Try to use spare capacity of local group without overloading it or
9665 * In some cases, the group's utilization is max or even
9668 * waiting task in this overloaded busiest group. Let's
9707 * busiest group
9719 * If the local group is more loaded than the selected
9720 * busiest group don't try to pull any tasks.
9729 * Both group are or will become overloaded and we're trying to get all
9733 * reduce the group load below the group capacity. Thus we look for
9745 * Decision matrix according to the local and busiest group type:
9765 * find_busiest_group - Returns the busiest group within the sched_domain
9773 * Return: - The busiest group if imbalance exists.
9799 /* There is no busy sibling group to pull tasks from */
9815 * If the busiest group is imbalanced the below checks don't
9824 * If the local group is busier than the selected busiest group
9837 * If the local group is more loaded than the selected
9838 * busiest group don't try to pull any tasks.
9848 * Don't pull any tasks if this group is already above the
9856 * If the busiest group is more loaded, use imbalance_pct to be
9873 * If the busiest group is not overloaded (and as a
9882 * If the busiest group is not overloaded
9884 * group wrt idle CPUs, it is balanced. The imbalance
9887 * on another group. Of course this applies only if
9888 * there is more than 1 CPU per group.
9912 * find_busiest_queue - find the busiest runqueue among the CPUs in the group.
9914 static struct rq *find_busiest_queue(struct lb_env *env, struct sched_group *group)
9921 for_each_cpu_and(i, sched_group_span(group), env->cpus)
9943 * the next pass will adjust the group classification and
10143 /* Are we the first CPU of this group ? */
10156 struct sched_group *group;
10183 group = find_busiest_group(&env);
10184 if (!group) {
10189 busiest = find_busiest_queue(&env, group);
10206 * an imbalance but busiest->nr_running <= 1, the group is
10300 * destination group that is receiving any migrated
10661 * CPU in our sched group which is doing load balancing more
11770 /* Account for a task changing its policy or group.
11981 /* guarantee group entities always have weight */