162306a36Sopenharmony_ciWhat: /sys/class/rtrs-client 262306a36Sopenharmony_ciDate: Feb 2020 362306a36Sopenharmony_ciKernelVersion: 5.7 462306a36Sopenharmony_ciContact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> 562306a36Sopenharmony_ciDescription: When a user of RTRS API creates a new session, a directory entry with 662306a36Sopenharmony_ci the name of that session is created under /sys/class/rtrs-client/<session-name>/ 762306a36Sopenharmony_ci 862306a36Sopenharmony_ciWhat: /sys/class/rtrs-client/<session-name>/add_path 962306a36Sopenharmony_ciDate: Feb 2020 1062306a36Sopenharmony_ciKernelVersion: 5.7 1162306a36Sopenharmony_ciContact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> 1262306a36Sopenharmony_ciDescription: RW, adds a new path (connection) to an existing session. Expected format is the 1362306a36Sopenharmony_ci following:: 1462306a36Sopenharmony_ci 1562306a36Sopenharmony_ci <[source addr,]destination addr> 1662306a36Sopenharmony_ci *addr ::= [ ip:<ipv4|ipv6> | gid:<gid> ] 1762306a36Sopenharmony_ci 1862306a36Sopenharmony_ciWhat: /sys/class/rtrs-client/<session-name>/max_reconnect_attempts 1962306a36Sopenharmony_ciDate: Feb 2020 2062306a36Sopenharmony_ciKernelVersion: 5.7 2162306a36Sopenharmony_ciContact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> 2262306a36Sopenharmony_ciDescription: Maximum number reconnect attempts the client should make before giving up 2362306a36Sopenharmony_ci after connection breaks unexpectedly. 2462306a36Sopenharmony_ci 2562306a36Sopenharmony_ciWhat: /sys/class/rtrs-client/<session-name>/mp_policy 2662306a36Sopenharmony_ciDate: Feb 2020 2762306a36Sopenharmony_ciKernelVersion: 5.7 2862306a36Sopenharmony_ciContact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> 2962306a36Sopenharmony_ciDescription: Multipath policy specifies which path should be selected on each IO: 3062306a36Sopenharmony_ci 3162306a36Sopenharmony_ci round-robin (0): 3262306a36Sopenharmony_ci select path in per CPU round-robin manner. 3362306a36Sopenharmony_ci 3462306a36Sopenharmony_ci min-inflight (1): 3562306a36Sopenharmony_ci select path with minimum inflights. 3662306a36Sopenharmony_ci 3762306a36Sopenharmony_ci min-latency (2): 3862306a36Sopenharmony_ci select path with minimum latency. 3962306a36Sopenharmony_ci 4062306a36Sopenharmony_ciWhat: /sys/class/rtrs-client/<session-name>/paths/ 4162306a36Sopenharmony_ciDate: Feb 2020 4262306a36Sopenharmony_ciKernelVersion: 5.7 4362306a36Sopenharmony_ciContact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> 4462306a36Sopenharmony_ciDescription: Each path belonging to a given session is listed here by its source and 4562306a36Sopenharmony_ci destination address. When a new path is added to a session by writing to 4662306a36Sopenharmony_ci the "add_path" entry, a directory <src@dst> is created. 4762306a36Sopenharmony_ci 4862306a36Sopenharmony_ciWhat: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/state 4962306a36Sopenharmony_ciDate: Feb 2020 5062306a36Sopenharmony_ciKernelVersion: 5.7 5162306a36Sopenharmony_ciContact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> 5262306a36Sopenharmony_ciDescription: RO, Contains "connected" if the session is connected to the peer and fully 5362306a36Sopenharmony_ci functional. Otherwise the file contains "disconnected" 5462306a36Sopenharmony_ci 5562306a36Sopenharmony_ciWhat: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/reconnect 5662306a36Sopenharmony_ciDate: Feb 2020 5762306a36Sopenharmony_ciKernelVersion: 5.7 5862306a36Sopenharmony_ciContact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> 5962306a36Sopenharmony_ciDescription: Write "1" to the file in order to reconnect the path. 6062306a36Sopenharmony_ci Operation is blocking and returns 0 if reconnect was successful. 6162306a36Sopenharmony_ci 6262306a36Sopenharmony_ciWhat: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/disconnect 6362306a36Sopenharmony_ciDate: Feb 2020 6462306a36Sopenharmony_ciKernelVersion: 5.7 6562306a36Sopenharmony_ciContact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> 6662306a36Sopenharmony_ciDescription: Write "1" to the file in order to disconnect the path. 6762306a36Sopenharmony_ci Operation blocks until RTRS path is disconnected. 6862306a36Sopenharmony_ci 6962306a36Sopenharmony_ciWhat: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/remove_path 7062306a36Sopenharmony_ciDate: Feb 2020 7162306a36Sopenharmony_ciKernelVersion: 5.7 7262306a36Sopenharmony_ciContact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> 7362306a36Sopenharmony_ciDescription: Write "1" to the file in order to disconnected and remove the path 7462306a36Sopenharmony_ci from the session. Operation blocks until the path is disconnected 7562306a36Sopenharmony_ci and removed from the session. 7662306a36Sopenharmony_ci 7762306a36Sopenharmony_ciWhat: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/hca_name 7862306a36Sopenharmony_ciDate: Feb 2020 7962306a36Sopenharmony_ciKernelVersion: 5.7 8062306a36Sopenharmony_ciContact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> 8162306a36Sopenharmony_ciDescription: RO, Contains the name of HCA the connection established on. 8262306a36Sopenharmony_ci 8362306a36Sopenharmony_ciWhat: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/hca_port 8462306a36Sopenharmony_ciDate: Feb 2020 8562306a36Sopenharmony_ciKernelVersion: 5.7 8662306a36Sopenharmony_ciContact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> 8762306a36Sopenharmony_ciDescription: RO, Contains the port number of active port traffic is going through. 8862306a36Sopenharmony_ci 8962306a36Sopenharmony_ciWhat: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/src_addr 9062306a36Sopenharmony_ciDate: Feb 2020 9162306a36Sopenharmony_ciKernelVersion: 5.7 9262306a36Sopenharmony_ciContact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> 9362306a36Sopenharmony_ciDescription: RO, Contains the source address of the path 9462306a36Sopenharmony_ci 9562306a36Sopenharmony_ciWhat: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/dst_addr 9662306a36Sopenharmony_ciDate: Feb 2020 9762306a36Sopenharmony_ciKernelVersion: 5.7 9862306a36Sopenharmony_ciContact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> 9962306a36Sopenharmony_ciDescription: RO, Contains the destination address of the path 10062306a36Sopenharmony_ci 10162306a36Sopenharmony_ciWhat: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/cur_latency 10262306a36Sopenharmony_ciDate: Feb 2020 10362306a36Sopenharmony_ciKernelVersion: 5.7 10462306a36Sopenharmony_ciContact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> 10562306a36Sopenharmony_ciDescription: RO, Contains the latency time calculated by the heart-beat messages. 10662306a36Sopenharmony_ci Whenever the client sends heart-beat message, it checks the time gap 10762306a36Sopenharmony_ci between sending the heart-beat message and receiving the ACK. 10862306a36Sopenharmony_ci This value can be changed regularly. 10962306a36Sopenharmony_ci 11062306a36Sopenharmony_ciWhat: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/stats/reset_all 11162306a36Sopenharmony_ciDate: Feb 2020 11262306a36Sopenharmony_ciKernelVersion: 5.7 11362306a36Sopenharmony_ciContact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> 11462306a36Sopenharmony_ciDescription: RW, Read will return usage help, write 0 will clear all the statistics. 11562306a36Sopenharmony_ci 11662306a36Sopenharmony_ciWhat: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/stats/cpu_migration 11762306a36Sopenharmony_ciDate: Feb 2020 11862306a36Sopenharmony_ciKernelVersion: 5.7 11962306a36Sopenharmony_ciContact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> 12062306a36Sopenharmony_ciDescription: RTRS expects that each HCA IRQ is pinned to a separate CPU. If it's 12162306a36Sopenharmony_ci not the case, the processing of an I/O response could be processed on a 12262306a36Sopenharmony_ci different CPU than where it was originally submitted. This file shows 12362306a36Sopenharmony_ci how many interrupts where generated on a non expected CPU. 12462306a36Sopenharmony_ci 12562306a36Sopenharmony_ci "from:" 12662306a36Sopenharmony_ci is the CPU on which the IRQ was expected, but not generated. 12762306a36Sopenharmony_ci "to:" 12862306a36Sopenharmony_ci is the CPU on which the IRQ was generated, but not expected. 12962306a36Sopenharmony_ci 13062306a36Sopenharmony_ciWhat: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/stats/reconnects 13162306a36Sopenharmony_ciDate: Feb 2020 13262306a36Sopenharmony_ciKernelVersion: 5.7 13362306a36Sopenharmony_ciContact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> 13462306a36Sopenharmony_ciDescription: Contains 2 unsigned int values, the first one records number of successful 13562306a36Sopenharmony_ci reconnects in the path lifetime, the second one records number of failed 13662306a36Sopenharmony_ci reconnects in the path lifetime. 13762306a36Sopenharmony_ci 13862306a36Sopenharmony_ciWhat: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/stats/rdma 13962306a36Sopenharmony_ciDate: Feb 2020 14062306a36Sopenharmony_ciKernelVersion: 5.7 14162306a36Sopenharmony_ciContact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> 14262306a36Sopenharmony_ciDescription: Contains statistics regarding rdma operations and inflight operations. 14362306a36Sopenharmony_ci The output consists of 6 values:: 14462306a36Sopenharmony_ci 14562306a36Sopenharmony_ci <read-count> <read-total-size> <write-count> \ 14662306a36Sopenharmony_ci <write-total-size> <inflights> <failovered> 147