162306a36Sopenharmony_ciWhat: /sys/class/rnbd-client 262306a36Sopenharmony_ciDate: Feb 2020 362306a36Sopenharmony_ciKernelVersion: 5.7 462306a36Sopenharmony_ciContact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> 562306a36Sopenharmony_ciDescription: Provide information about RNBD-client. 662306a36Sopenharmony_ci All sysfs files that are not read-only provide the usage information on read: 762306a36Sopenharmony_ci 862306a36Sopenharmony_ci Example:: 962306a36Sopenharmony_ci 1062306a36Sopenharmony_ci # cat /sys/class/rnbd-client/ctl/map_device 1162306a36Sopenharmony_ci 1262306a36Sopenharmony_ci > Usage: echo "sessname=<name of the rtrs session> path=<[srcaddr,]dstaddr> 1362306a36Sopenharmony_ci > [path=<[srcaddr,]dstaddr>] device_path=<full path on remote side> 1462306a36Sopenharmony_ci > [access_mode=<ro|rw|migration>] > map_device 1562306a36Sopenharmony_ci > 1662306a36Sopenharmony_ci > addr ::= [ ip:<ipv4> | ip:<ipv6> | gid:<gid> ] 1762306a36Sopenharmony_ci 1862306a36Sopenharmony_ciWhat: /sys/class/rnbd-client/ctl/map_device 1962306a36Sopenharmony_ciDate: Feb 2020 2062306a36Sopenharmony_ciKernelVersion: 5.7 2162306a36Sopenharmony_ciContact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> 2262306a36Sopenharmony_ciDescription: Expected format is the following:: 2362306a36Sopenharmony_ci 2462306a36Sopenharmony_ci sessname=<name of the rtrs session> 2562306a36Sopenharmony_ci path=<[srcaddr,]dstaddr> [path=<[srcaddr,]dstaddr> ...] 2662306a36Sopenharmony_ci device_path=<full path on remote side> 2762306a36Sopenharmony_ci [access_mode=<ro|rw|migration>] 2862306a36Sopenharmony_ci 2962306a36Sopenharmony_ci Where: 3062306a36Sopenharmony_ci 3162306a36Sopenharmony_ci sessname: 3262306a36Sopenharmony_ci accepts a string not bigger than 256 chars, which identifies 3362306a36Sopenharmony_ci a given session on the client and on the server. 3462306a36Sopenharmony_ci I.e. "clt_hostname-srv_hostname" could be a natural choice. 3562306a36Sopenharmony_ci 3662306a36Sopenharmony_ci path: 3762306a36Sopenharmony_ci describes a connection between the client and the server by 3862306a36Sopenharmony_ci specifying destination and, when required, the source address. 3962306a36Sopenharmony_ci The addresses are to be provided in the following format:: 4062306a36Sopenharmony_ci 4162306a36Sopenharmony_ci ip:<IPv6> 4262306a36Sopenharmony_ci ip:<IPv4> 4362306a36Sopenharmony_ci gid:<GID> 4462306a36Sopenharmony_ci 4562306a36Sopenharmony_ci for example:: 4662306a36Sopenharmony_ci 4762306a36Sopenharmony_ci path=ip:10.0.0.66 4862306a36Sopenharmony_ci 4962306a36Sopenharmony_ci The single addr is treated as the destination. 5062306a36Sopenharmony_ci The connection will be established to this server from any client IP address. 5162306a36Sopenharmony_ci 5262306a36Sopenharmony_ci :: 5362306a36Sopenharmony_ci 5462306a36Sopenharmony_ci path=ip:10.0.0.66,ip:10.0.1.66 5562306a36Sopenharmony_ci 5662306a36Sopenharmony_ci First addr is the source address and the second is the destination. 5762306a36Sopenharmony_ci 5862306a36Sopenharmony_ci If multiple "path=" options are specified multiple connection 5962306a36Sopenharmony_ci will be established and data will be sent according to 6062306a36Sopenharmony_ci the selected multipath policy (see RTRS mp_policy sysfs entry description). 6162306a36Sopenharmony_ci 6262306a36Sopenharmony_ci device_path: 6362306a36Sopenharmony_ci Path to the block device on the server side. Path is specified 6462306a36Sopenharmony_ci relative to the directory on server side configured in the 6562306a36Sopenharmony_ci 'dev_search_path' module parameter of the rnbd_server. 6662306a36Sopenharmony_ci The rnbd_server prepends the <device_path> received from client 6762306a36Sopenharmony_ci with <dev_search_path> and tries to open the 6862306a36Sopenharmony_ci <dev_search_path>/<device_path> block device. On success, 6962306a36Sopenharmony_ci a /dev/rnbd<N> device file, a /sys/block/rnbd<N>/ 7062306a36Sopenharmony_ci directory and an entry in /sys/class/rnbd-client/ctl/devices 7162306a36Sopenharmony_ci will be created. 7262306a36Sopenharmony_ci 7362306a36Sopenharmony_ci If 'dev_search_path' contains '%SESSNAME%', then each session can 7462306a36Sopenharmony_ci have different devices namespace, e.g. server was configured with 7562306a36Sopenharmony_ci the following parameter "dev_search_path=/run/rnbd-devs/%SESSNAME%", 7662306a36Sopenharmony_ci client has this string "sessname=blya device_path=sda", then server 7762306a36Sopenharmony_ci will try to open: /run/rnbd-devs/blya/sda. 7862306a36Sopenharmony_ci 7962306a36Sopenharmony_ci access_mode: 8062306a36Sopenharmony_ci the access_mode parameter specifies if the device is to be 8162306a36Sopenharmony_ci mapped as "ro" read-only or "rw" read-write. The server allows 8262306a36Sopenharmony_ci a device to be exported in rw mode only once. The "migration" 8362306a36Sopenharmony_ci access mode has to be specified if a second mapping in read-write 8462306a36Sopenharmony_ci mode is desired. 8562306a36Sopenharmony_ci 8662306a36Sopenharmony_ci By default "rw" is used. 8762306a36Sopenharmony_ci 8862306a36Sopenharmony_ci nr_poll_queues 8962306a36Sopenharmony_ci specifies the number of poll-mode queues. If the IO has HIPRI flag, 9062306a36Sopenharmony_ci the block-layer will send the IO via the poll-mode queue. 9162306a36Sopenharmony_ci For fast network and device the polling is faster than interrupt-base 9262306a36Sopenharmony_ci IO handling because it saves time for context switching, switching to 9362306a36Sopenharmony_ci another process, handling the interrupt and switching back to the 9462306a36Sopenharmony_ci issuing process. 9562306a36Sopenharmony_ci 9662306a36Sopenharmony_ci Set -1 if you want to set it as the number of CPUs 9762306a36Sopenharmony_ci By default rnbd client creates only irq-mode queues. 9862306a36Sopenharmony_ci 9962306a36Sopenharmony_ci NOTICE: MUST make a unique session for a device using the poll-mode queues. 10062306a36Sopenharmony_ci 10162306a36Sopenharmony_ci Exit Codes: 10262306a36Sopenharmony_ci 10362306a36Sopenharmony_ci If the device is already mapped it will fail with EEXIST. If the input 10462306a36Sopenharmony_ci has an invalid format it will return EINVAL. If the device path cannot 10562306a36Sopenharmony_ci be found on the server, it will fail with ENOENT. 10662306a36Sopenharmony_ci 10762306a36Sopenharmony_ci Finding device file after mapping 10862306a36Sopenharmony_ci --------------------------------- 10962306a36Sopenharmony_ci 11062306a36Sopenharmony_ci After mapping, the device file can be found by: 11162306a36Sopenharmony_ci o The symlink /sys/class/rnbd-client/ctl/devices/<device_id>@<session_name> 11262306a36Sopenharmony_ci points to /sys/block/<dev-name>. The last part of the symlink destination 11362306a36Sopenharmony_ci is the same as the device name. By extracting the last part of the 11462306a36Sopenharmony_ci path the path to the device /dev/<dev-name> can be build. 11562306a36Sopenharmony_ci 11662306a36Sopenharmony_ci * /dev/block/$(cat /sys/class/rnbd-client/ctl/devices/<device_id>@<session_name>/dev) 11762306a36Sopenharmony_ci 11862306a36Sopenharmony_ci How to find the <device_id> of the device is described on the next 11962306a36Sopenharmony_ci section. 12062306a36Sopenharmony_ci 12162306a36Sopenharmony_ciWhat: /sys/class/rnbd-client/ctl/devices/ 12262306a36Sopenharmony_ciDate: Feb 2020 12362306a36Sopenharmony_ciKernelVersion: 5.7 12462306a36Sopenharmony_ciContact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> 12562306a36Sopenharmony_ciDescription: For each device mapped on the client a new symbolic link is created as 12662306a36Sopenharmony_ci /sys/class/rnbd-client/ctl/devices/<device_id>@<session_name>, which points 12762306a36Sopenharmony_ci to the block device created by rnbd (/sys/block/rnbd<N>/). 12862306a36Sopenharmony_ci The <device_id> of each device is created as follows: 12962306a36Sopenharmony_ci 13062306a36Sopenharmony_ci - If the 'device_path' provided during mapping contains slashes ("/"), 13162306a36Sopenharmony_ci they are replaced by exclamation mark ("!") and used as as the 13262306a36Sopenharmony_ci <device_id>. Otherwise, the <device_id> will be the same as the 13362306a36Sopenharmony_ci "device_path" provided. 134