Test Specification

This file lists all test cases’ specifications.

class gpu
Description
This test suite runs the gpu test cases.
Platform

Azure, Ready

Area

gpu

Category

functional

validate_load_gpu_driver()
Description
This test case verifies if gpu drivers are loaded fine.
Steps:
1. Validate if the VM SKU is supported for GPU.
2. Install LIS drivers if not already installed for Fedora and its
derived distros. Reboot the node
3. Install required gpu drivers on the VM and reboot the node. Validate gpu
drivers can be loaded successfully.
Priority

1

Requirement

supported_features[Gpu, SerialConsole]

validate_gpu_adapter_count()
Description
This test case verifies the gpu adapter count.
Steps:
1. Assert that node supports GPU.
2. If GPU modules are not loaded, install and load the module first.
3. Find the expected gpu count for the node.
4. Validate expected and actual gpu count using lsvmbus output.
5. Validate expected and actual gpu count using lspci output.
6. Validate expected and actual gpu count using gpu vendor commands
example - nvidia-smi
Priority

2

Requirement

supported_features[Gpu]

gpu_rescind_validation()
Description
This test case will
1. Validate disabling GPU devices.
2. Validate enable back the disabled GPU devices.
Priority

2

Requirement

supported_features[Gpu]

class Nested
Description
This test suite is used to run nested vm related tests.
Platform

Azure, Ready

Area

nested

Category

functional

verify_nested_basic()
Description
This test case will run basic tests on provisioned L2 vm.
Steps:
1. Create L2 VM with Qemu.
2. Verify that files can be copied from L1 VM to L2 VM.
3. Verify that files from internet can be downloaded to L2 VM.
Priority

1

class Kvp
Description
This test suite verify the KVP service runs well on Azure and Hyper-V
platforms. The KVP is used to communicate between Windows host and guest VM.
Platform

Azure, Ready

Area

kvp

Category

functional

verify_kvp()
Description
Verify KVP daemon installed, running, permission correct.
1. verify that the KVP Daemon is running.
2. run the KVP client tool and verify that the data pools are created and
accessible.
3. check kvp_pool file permission is 644.
4. check kernel version supports hv_kvp.
5. Check if KVP pool 3 file has a size greater than zero.
6. At least 11 items are present in pool 3, and verify record count is
correct.
Priority

1

class Boot
Description
This test suite is to test VM working well after updating on VM and rebooting.
Platform

Azure, Ready

Area

core

Category

functional

verify_boot_with_debug_kernel()
Description
This test case will
1. Skip testing if the distro is not redhat/centos type, RHEL added this test
case, since they encounter an issue which is seeing call trace when boot
with debug kernel.
2. Install kernel-debug package and set boot with this debug kernel.
3. Reboot VM, check kernel version is debug type.
Priority

3

Requirement

supported_features[SerialConsole]

class Storage
Description
This test suite is used to run storage related tests.
Platform

Azure, Ready

Area

storage

Category

functional

verify_swap()
Description
This test will check that the swap is correctly configured on the VM.
Steps:
1. Check if swap file/partition is configured by checking the output of
swapon -s and lsblk.
2. Check swap status in waagent.conf.
3. Verify that truth value in step 1 and step 2 match.
Priority

1

Requirement

supported_platform_type[AZURE]

verify_os_partition_identifier()
Description
This test will verify that identifier of root partition matches
from different sources.
Steps:
1. Get the partition identifier from blkid command.
2. Verify that the partition identifier from blkid is present in dmesg.
3. Verify that the partition identifier from blkid is present in fstab output.
Priority

1

Requirement

supported_platform_type[AZURE]

verify_root_device_timeout_setting()
Description
This test will check that VM root disk(os disk) is provisioned
with the correct timeout.
Steps:
1. Find the root disk (os disk) for the VM. The root disk
is the entry with mount point /’ in the output of `mount command.
2. Verify the timeout value for root disk in
/sys/block/<partition>/device/timeout file is set to 300.
Priority

1

Requirement

supported_platform_type[AZURE]

verify_resource_disk_mtab_entry()
Description
This test will check that the resource disk is present in the list of mounted
devices. Most VMs contain a resource disk, which is not a managed disk and
provides short-term storage for applications and processes. It is intended to
only store data such as page or swap files.
Steps:
1. Get the mount point for the resource disk. If /var/log/cloud-init.log
file is present, mount location is /mnt, otherwise it is obtained from
ResourceDisk.MountPoint entry in waagent.conf configuration file.
2. Verify that “/dev/<disk> <mount_point>` entry is present in
/etc/mtab file and the disk should not be equal to os disk.
Priority

1

Requirement

supported_platform_type[AZURE]

verify_resource_disk_io()
Description
This test will check that the file IO operations are working correctly
Steps:
1. Get the mount point for the resource disk. If /var/log/cloud-init.log
file is present, mount location is /mnt, otherwise it is obtained from
ResourceDisk.MountPoint entry in waagent.conf configuration file.
2. Verify that resource disk is mounted from the output of mount command.
3. Write a text file to the resource disk.
4. Read the text file and verify that content is same.
Priority

1

Requirement

supported_platform_type[AZURE]

class Dhcp
Description
This test suite covers DHCP functionalities.
Platform

Azure, Ready

Area

core

Category

functional

verify_dhcp_client_timeout()
Description
This test case check the timeout setting of DHCP on Azure equals or more
than 300 seconds.
Priority

1

Requirement

supported_platform_type[AZURE, READY]

class AzureImageStandard
Description
This test suite is used to check azure image configuration.
Platform

Azure, Ready

Area

azure_image_standard

Category

functional

verify_dhcp_file_configuration()
Description
This test will verify that dhcp file exists at
/etc/sysconfig/network/dhcp and DHCLIENT_SET_HOSTNAME is set
to no.
Steps:
1. Verify that dhcp file exists.
2. Verify that DHCLIENT_SET_HOSTNAME=”no” is present in the file.
Priority

1

verify_default_targetpw()
Description
This test will verify that Defaults targetpw is not enabled in the
/etc/sudoers file.
If targetpw is set, sudo will prompt for the
password of the user specified by the -u option (defaults to root)
instead of the password of the invoking user when running a command
or editing a file. More information can be found here :
Steps:
1. Get the content of /etc/sudoers file.
2. Verify that Defaults targetpw should be disabled, if present.
Priority

1

verify_network_file_configuration()
Description
This test will verify that network file exists in /etc/sysconfig and networking
is enabled on Fedora based distros.
Steps:
1. Verify that network file exists.
2. Verify that networking is enabled in the file.
Priority

1

verify_network_manager_not_installed()
Description
This test will verify that network manager doesn’t conflict with the
waagent on Fedora based distros.
Steps:
1. Get the output of command rpm -q NetworkManager and verify that
network manager is not installed.
Priority

1

verify_repository_installed()
Description
This test will check that repositories are correctly installed.
Steps:
1. Verify the repository configuration depending on the distro type.
Priority

1

verify_ifcfg_eth0()
Description
This test will verify contents of ifcfg-eth0 file on Fedora based distros.
Steps:
1. Read the ifcfg-eth0 file and verify that “DEVICE=eth0”, “BOOTPROTO=dhcp” and
“ONBOOT=yes” is present in network file.
Priority

1

verify_hv_kvp_daemon_installed()
Description
This test will check that hv-kvp-daemon-init is installed on Debian based
distros. This is an optional requirement.
Steps:
1. Verify that list of running process matching name hv_kvp_daemon
has length greater than zero.
Priority

1

verify_udev_rules_moved()
Description
This test will verify that udev rules have been moved out in CoreOS
and Fedora based distros
Steps:
1. Verify that 75-persistent-net-generator.rules and 70-persistent-net.rules
files are not present.
Priority

1

verify_yum_conf()
Description
This test will verify content of yum.conf file on Fedora based distros
for version < 6.6.0
Steps:
1. Read the yum.conf file and verify that “http_caching=packages” is
present in the file.
Priority

1

verify_grub()
Description
This test will check the configuration of the grub file and verify that numa
is disabled for Redhat distro version < 6.6.0
Steps:
1. Verify grub configuration depending on the distro type.
2. For Redhat based distros, verify that numa is disabled for versions < 6.6.0
Priority

1

class Provisioning
Description
This test suite uses to verify if an environment can be provisioned correct or not.
- The basic smoke test can run on all images to determinate if a image can boot and
reboot.
- Other provisioning tests verify if an environment can be provisioned with special
hardware configurations.
Platform

Azure, Ready

Area

provisioning

Category

functional

smoke_test()
Description
This case verifies whether a node is operating normally.
Steps,
1. Connect to TCP port 22. If it’s not connectable, failed and check whether
there is kernel panic.
2. Connect to SSH port 22, and reboot the node. If there is an error and kernel
panic, fail the case. If it’s not connectable, also fail the case.
3. If there is another error, but not kernel panic or tcp connection, pass with
warning.
4. Otherwise, fully passed.
Priority

0

Requirement

supported_features[SerialConsole]

verify_deployment_provision_ephemeral_managed_disk()
Description
This case runs smoke test on a node provisioned with ephemeral disk.
The test steps are same as smoke_test.
Priority

1

Requirement

supported_features[SerialConsole]

verify_reboot_in_platform()
Description
This case runs smoke test on a node provisioned.
The test steps are almost the same as smoke_test except for
executing reboot from Azure SDK.
Priority

1

Requirement

supported_features[SerialConsole, StartStop]

verify_deployment_provision_premium_disk()
Description
This case runs smoke test on a node provisioned with premium disk.
The test steps are same as smoke_test.
Priority

1

Requirement

supported_features[SerialConsole]

verify_deployment_provision_sriov()
Description
This case runs smoke test on a node provisioned with sriov.
The test steps are same as smoke_test.
Priority

1

Requirement

supported_features[SerialConsole]

verify_deployment_provision_synthetic_nic()
Description
This case runs smoke test on a node provisioned with synthetic nic.
The test steps are same as smoke_test.
Priority

1

Requirement

supported_features[SerialConsole]

verify_stop_start_in_platform()
Description
This case runs smoke test on a node provisioned.
The test steps are almost the same as smoke_test except for
executing stop then start from Azure SDK.
Priority

2

Requirement

supported_features[SerialConsole, StartStop]

verify_deployment_provision_standard_ssd_disk()
Description
This case runs smoke test on a node provisioned with standard ssd disk.
The test steps are same as smoke_test.
Priority

1

Requirement

supported_features[SerialConsole]

class HvModule
Description
This test suite covers test cases previously handled by LISAv2:
LIS-MODULES-CHECK, VERIFY-LIS-MODULES-VERSION,
INITRD-MODULES-CHECK, RELOAD-MODULES-SMP
It is responsible for ensuring the Hyper V drivers are all present,
are included in initrd, and are all the same version.
Platform

Azure, Ready

Area

core

Category

functional

verify_lis_modules_version()
Description
This test case will
1. Verify the list of given LIS kernel modules and verify if the version
matches with the Linux kernel release number. (Drivers loaded directly in
to the kernel are skipped)
Priority

1

reload_hyperv_modules()
Description
This test case will reload hyper-v modules as a stress test.
Priority

1

verify_initrd_modules()
Description
This test case will ensure all necessary hv_modules are present in
initrd. This is achieved by
1. Skipping any modules that are loaded directly in the kernel
2. Find the path of initrd file
3. Use lsinitrd tool to check whether a necessary module is missing
Priority

2

verify_hyperv_modules()
Description
This test case will
1. Verify the presence of all Hyper V drivers using lsmod
to look for the drivers not directly loaded into the kernel.
Priority

1

class Floppy
Description
This test suite ensures the floppy driver is disabled.
The floppy driver is not needed on Azure and
is known to cause problems in some scenarios.
Platform

Azure, Ready

Area

core

Category

functional

check_floppy_module()
Description
The goal of this test is to ensure the floppy module is not enabled
for images used on the Azure platform.
This test case will
1. Dry-run modprobe to see if floppy module can be loaded
2. If “insmod” would be executed then the module is not already loaded
3. If module cannot be found then it is not loaded
If the module is loaded, running modprobe will have no output
Priority

1

class TimeSync
Description
This test suite is related with time sync.
Platform

Azure, Ready

Area

time

Category

functional

timesync_validate_ptp()
Description
This test is to check -
1. PTP time source is available on Azure guests (newer versions of Linux).
2. PTP device name is hyperv.
3. When accelerated network is enabled, multiple PTP devices will
be available, the names of ptp are changeable, create the symlink
/dev/ptp_hyperv to whichever /dev/ptp entry corresponds to the Azure host.
4. Chrony should be configured to use the symlink /dev/ptp_hyperv
instead of /dev/ptp0 or /dev/ptp1.
Priority

2

timesync_ntp()
Description
This test is to check, ntp works properly.
1. Stop systemd-timesyncd if this service exists.
2. Set rtc clock to system time.
3. Restart Ntp service.
4. Check and set server setting in config file.
5. Restart Ntp service to reload with new config.
6. Check leap code using ntpq -c rv.
7. Check local time is synchronised with time server using ntpstat.
Priority

2

timesync_check_unbind_clockevent()
Description
This test is to check -
1. Current clock event name is ‘Hyper-V clockevent’.
2. ‘Hyper-V clockevent’ and ‘hrtimer_interrupt’ show up times in
/proc/timer_list should equal to cpu count.
3. when cpu count is 1 and cpu type is Intel type, unbind current time
clock event, check current time clock event switch to ‘lapic’.
Priority

2

timesync_chrony()
Description
This test is to check chrony works properly.
1. Restart chrony service.
2. Check and set server setting in config file.
3. Restart chrony service to reload with new config.
4. Check chrony sources and sourcestats.
5. Check chrony tracking.
Priority

2

timesync_check_unbind_clocksource()
Description
This test is to check -
1. Check clock source name is one of hyperv_clocksource_tsc_page,
lis_hv_clocksource_tsc_page, hyperv_clocksource.
2. Check CPU flag contains constant_tsc from /proc/cpuinfo.
3. Check clocksource name shown up in dmesg.
4. Unbind current clock source if there are 2+ clock sources, check current
clock source can be switched to a different one.
Priority

2

class LsVmBus
Description
This test suite is used to check vmbus devices and their associated vmbus channels.
Platform

Azure, Ready

Area

core

Category

functional

lsvmbus_count_devices_channels()
Description
This test case will
1. Check expected vmbus device names presented in the lsvmbus output.
- Operating system shutdown
- Time Synchronization
- Heartbeat
- Data Exchange
- Synthetic mouse
- Synthetic keyboard
- Synthetic network adapter
- Synthetic SCSI Controller
- Synthetic IDE Controller (gen1 only)
2. Check that each netvsc and storvsc SCSI device have correct number of vmbus
channels created and associated.
2.1 Check expected channel count of each netvsc is min (num of vcpu, 8).
2.2 Check expected channel count of each storvsc SCSI device is min (num of
vcpu/4, 64).
2.2.1 Caculate channel count of each storvsc SCSI device.
2.2.2 Cap of channel count of each storvsc SCSI device,
it is decided by host storage VSP driver.
Priority

1

class Dpdk
Description
This test suite check DPDK functionality
Platform

Azure, Ready

Area

dpdk

Category

functional

verify_dpdk_send_receive_failsafe()
Description
Tests a basic sender/receiver setup for default failsafe driver setup.
Sender sends the packets, receiver receives them.
We check both to make sure the received traffic is within the expected
order-of-magnitude.
Priority

2

Requirement

min_count=2

verify_dpdk_build_netvsc()
Description
netvsc direct pmd version.
This test case checks DPDK can be built and installed correctly.
Prerequisites, accelerated networking must be enabled.
The VM should have at least two network interfaces,
with one interface for management.
Priority

2

Requirement

network_interface

verify_dpdk_sriov_rescind_failover_send_only()
Description
test sriov failsafe during vf revoke (send only version)
Priority

2

Requirement

network_interface

verify_dpdk_build_failsafe()
Description
failsafe (azure default, recommended) version.
This test case checks DPDK can be built and installed correctly.
Prerequisites, accelerated networking must be enabled.
The VM should have at least two network interfaces,
with one interface for management.
Priority

2

Requirement

network_interface

verify_dpdk_ring_ping()
Description
This test runs the dpdk ring ping utility from:
to measure the maximum latency for 99.999 percent of packets during
the test run. The maximum should be under 200000 nanoseconds
(.2 milliseconds).
Not dependent on any specific PMD.
Priority

2

Requirement

network_interface

verify_dpdk_sriov_rescind_failover_receiver()
Description
test sriov failsafe during vf revoke (receive side)
Priority

2

Requirement

min_count=2

verify_dpdk_send_receive_netvsc()
Description
Tests a basic sender/receiver setup for direct netvsc pmd setup.
Sender sends the packets, receiver receives them.
We check both to make sure the received traffic is within the expected
order-of-magnitude.
Priority

2

Requirement

min_count=2

class InfinibandSuit
Description
Tests the functionality of infiniband.
Platform

Azure, Ready

Area

core

Category

functional

verify_hpc_over_sriov()
Description
This test case will
1. Determine whether the VM has Infiniband over SR-IOV
2. Ensure waagent is configures with OS.EnableRDMA=y
3. Check that appropriate drivers are present
Priority

2

Requirement

supported_features[Infiniband]

verify_hpc_over_nd()
Description
This test case will
1. Determine whether the VM has Infiniband over Network Direct
2. Ensure waagent is configures with OS.EnableRDMA=y
3. Check that appropriate drivers are present
Priority

2

Requirement

supported_features[Infiniband]

class KdumpCrash
Description
This test suite is used to verify if kernel crash dmup is effect, which is judged
through vmcore file is generated after triggering kdump by sysrq.
It has 7 test cases. They verfy if kdump is effect when:
1. VM has 1 cpu
2. VM has 2-8 cpus and trigger kdump on cpu 1
3. VM has 33-192 cpus and trigger kdump on cpu 32
4. VM has 193-415 cpus and trigger kdump on cpu 192
5. VM has more than 415 cpus and trigger kdump on cpu 415
6. crashkernel is set “auto”
7. crashkernel is set “auto” and VM has more than 2T memory
Platform

Azure, Ready

Area

kdump

Category

functional

kdumpcrash_validate_on_cpu192()
Description
This test case verfies if the kdump is effect when VM has 193~415 cpus, and
trigger kdump on the 193th cpu(cpu192), which is designed by a known issue.
The test steps are same as kdumpcrash_validate_single_core.
Priority

2

Requirement

node

kdumpcrash_validate_single_core()
Description
This test case verfies if kdump is effect when VM has 1 cpu.
VM need 2G memory at least to make sure it has enough memory to load crash
kernel.
Steps:
1. Check if vmbus version and kernel configurations support for crash dump.
2. Specify the memory reserved for crash kernel in kernel cmdline, setting the
“crashkernel” option to the required value.
a. Modify the grub config file to add crashkrenel option or change the
value to the required one. (For Redhat 8, no need to modify grub config
file. It can specify crashkernel by using grubby command directly)
b. Update grub config
4. If needed, config the dump path.
3. Reboot system to make kdump effect.
4. Check if the crash kernel is loaded.
a. Check if kernel cmdline has crashkernel option and the value is expected
b. Check if /sys/kernel/kexec_crash_loaded file exists and the value is ‘1’
c. Check if /proc/iomem is reserved memory for crash kernel
5. Trigger kdump through ‘echo c > /proc/sysrq-trigger’, or trigger on
specified CPU by using command “taskset -c”.
6. Check if vmcore is generated under the dump path we configure after system
boot up.
Priority

1

Requirement

node

kdumpcrash_validate_on_cpu32()
Description
This test case verfies if the kdump is effect when VM has 33~192 cpus and
trigger kdump on the 33th cpu(cpu32), which is designed by a known issue.
The test steps are same as kdumpcrash_validate_single_core.
Priority

2

Requirement

node

kdumpcrash_validate_on_cpu415()
Description
This test case verfies if the kdump is effect when VM has more than 415 cpus,
and trigger kdump on the 416th cpu(cpu415), which is designed by a known issue.
The test steps are same as kdumpcrash_validate_single_core.
Priority

2

Requirement

node

kdumpcrash_validate_smp()
Description
This test case verfies if the kdump is effect when VM has 2~8 cpus, and
trigger kdump on the second cpu(cpu1), which is designed by a known issue.
The test steps are same as kdumpcrash_validate_single_core.
Priority

2

Requirement

node

kdumpcrash_validate_auto_size()
Description
This test case verfies if the kdump is effect when crashkernel is set auto.
The test steps are same as kdumpcrash_validate_single_core.
Priority

2

kdumpcrash_validate_large_memory_auto_size()
Description
This test case verfies if the kdump is effect when crashkernel is set auto and
the memory is more than 2T. With the crashkernel=auto parameter, system will
reserved a suitable size memory for crash kernel. We want to see if the
crashkernel=auto can also handle this scenario when the system memory is large.
The test steps are same as kdumpcrash_validate_single_core.
Priority

2

Requirement

node

class docker
Description
This test suite runs the docker test cases for java, python, dotnet 3.1
, dotnet5.0, and wordpress.
Platform

Azure, Ready

Area

docker

Category

functional

verify_docker_java_app()
Description
This test case creates and runs a java app using docker
Steps:
1. Install Docker
2. Copy java dockerfile and program file to node
3. Create docker image and run docker container
4. Check results of docker run against java string identifier
Priority

2

verify_docker_dotnet50_app()
Description
This test case creates and runs a dotnet app using docker
Steps:
1. Install Docker
2. Copy dotnet dockerfile into node
3. Create docker image and run docker container
4. Check results of docker run against dotnet string identifier
Priority

2

verify_docker_dotnet31_app()
Description
This test case creates and runs a dotnet app using docker
Steps:
1. Install Docker
2. Copy dotnet dockerfile into node
3. Create docker image and run docker container
4. Check results of docker run against dotnet string identifier
Priority

1

verify_docker_compose_wordpress_app()
Description
This test case uses docker-compose to create and run a wordpress mysql app
Steps:
1. Install Docker and Docker-Compose on node
2. Copy docker-compose.yml into node
3. Start docker container with docker-compose
4. Run ps in the docker container and capture output
5. Check that “apache2” can be found in captured output
Priority

3

verify_docker_python_app()
Description
This test case creates and runs a python app using docker
Steps:
1. Install Docker
2. Copy python dockerfile and program file to node
3. Create docker image and run docker container
4. Check results of docker run against python string identifier
Priority

3

class nvme
Description
This test suite is to validate NVMe disk on Linux VM.
Platform

Azure, Ready

Area

nvme

Category

functional

nvme_basic_validation()
Description
This test case will
1. Get nvme devices and nvme namespaces from /dev/ folder,
compare the count of nvme namespaces and nvme devices.
2. Compare the count of nvme namespaces return from nvme list
and list nvme namespaces under /dev/.
3. Compare nvme devices count return from lspci
and list nvme devices under /dev/.
4. Azure platform only, nvme devices count should equal to
actual vCPU count / 8.
Priority

1

Requirement

supported_features[Nvme]

nvme_blkdiscard_validation()
Description
This test case will
1. Create a partition, xfs filesystem and mount it.
2. Umount the mountpoint.
3. Run blkdiscard command on the partition.
4. Remount command should fail after run blkdiscard command.
Priority

3

Requirement

supported_features[Nvme]

nvme_max_disk_validation()
Description
This case runs nvme_basic_validation test against 10 NVMe disks.
The test steps are same as nvme_basic_validation.
Priority

2

Requirement

supported_features

nvme_fstrim_validation()
Description
This test case will
1. Create a partition, xfs filesystem and mount it.
2. Check how much the mountpoint is trimmed before operations.
3. Create a 300 gb file ‘data’ using dd command in the partition.
4. Check how much the mountpoint is trimmed after creating the file.
5. Delete the file ‘data’.
6. Check how much the mountpoint is trimmed after deleting the file,
and compare the final fstrim status with initial fstrim status.
Priority

3

Requirement

supported_features[Nvme]

nvme_rescind_validation()
Description
This test case will
1. Disable NVME devices.
2. Enable NVME device.
Priority

2

Requirement

supported_features[Nvme]

nvme_function_validation()
Description
This test case will do following things for each NVMe device.
1. Get the number of errors from nvme-cli before operations.
2. Create a partition, filesystem and mount it.
3. Create a txt file on the partition, content is ‘TestContent’.
4. Create a file ‘data’ on the partition, get the md5sum value.
5. Umount and remount the partition.
6. Get the txt file content, compare the value.
7. Compare the number of errors from nvme-cli after operations.
Priority

2

Requirement

supported_features[Nvme]

nvme_sriov_rescind_validation()
Description
This test case does following steps to verify VM working normally during
disable and enable nvme and sriov devices.
1. Disable PCI devices.
2. Enable PCI devices.
3. Get PCI devices slots.
4. Check PCI devices are back after rescan.
Priority

2

Requirement

supported_features[Nvme]

nvme_manage_ns_validation()
Description
This test case will run commands 2-5, the commands are expected fail or not
based on the capabilities of the device.
1. Use nvme id-ctrl device command list the capabilities of the device.
1.1 When ‘Format NVM Supported’ shown up in output of ‘nvme id-ctrl device’,
then nvme disk can be format, otherwise, it can’t be format.
1.2 When ‘NS Management and Attachment Supported’ shown up in output of
‘nvme id-ctrl device’, nvme namespace can be created, deleted and detached,
otherwise it can’t be managed.
2. nvme format namespace - format a namespace.
3. nvme create-ns namespace - create a namespace.
4. nvme delete-ns -n 1 namespace - delete a namespace.
5. nvme detach-ns -n 1 namespace - detach a namespace.
Priority

3

Requirement

supported_features[Nvme]

class Lis
Description
This test suite contains tests that are dependent on an LIS driver
Platform

Azure, Ready

Area

lis

Category

functional

verify_lis_driver_version()
Description
Downloads header files based on the LIS version and compares the installed
LIS version to the expected one which is found in the header files.
Steps:
1. Check for RPM
2. Capture installed LIS version on the node
3. For each rhel version (5,6,7), it downloads the header file and compares
the LIS version in the header file with the LIS version installed
Priority

1

class NetworkSettings
Description
This test suite runs the ethtool related network test cases.
Platform

Azure, Ready

Area

network

Category

functional

validate_device_channels_change()
Description
This test case verifies changing device channels count with ethtool.
Steps:
1. Get the current device channels info.
2 a. Keep Changing the channel count from min to max value using ethtool.
b. Get the channel count info and validate the channel count
value is equal to the new value assigned.
3. Revert back the channel count to its original value.
Priority

1

validate_device_gro_lro_settings_change()
Description
This test case verifies changing device’s GRO and LRO setting takes
into affect.
Steps:
1. Get all the device’s generic-receive-offload and large-receive-offload
settings.
2. If both GRO and LRO settings are “[fixed]” then skip testing specific
device.
3. Try flipping the GRO and LRO settings and validate it takes affect.
4. Revert back the settings to original values.
Priority

1

validate_device_msg_level_change()
Description
This test case verifies whether setting/unsetting device’s
message level flag takes into affect.
Steps:
1. Verify Get/Set message level supported on kernel version.
2. Get all the device’s message level number and name setting.
2. Depending on current setting, set/unset a message flag by number
and name.
3. Validate changing the message level flag setting.
4. Revert back the setting to original value.
Priority

2

validate_device_enabled_features()
Description
This test case verifies required device features are enabled.
Steps:
1. Get the device’s enabled features.
2. Validate below features are in the list of enabled features-
rx-checksumming
tx-checksumming
tcp-segmentation-offload
scatter-gather
Priority

1

validate_ringbuffer_settings_change()
Description
This test case verifies if ring buffer settings can be changed with ethtool.
Steps:
1. Get the current ring buffer settings.
2. Change the rx and tx value to new_values using ethtool.
3. Get the settings again and validate the current rx and tx
values are equal to the new_values assigned.
4. Revert back the rx and tx value to their original values.
Priority

1

validate_device_rss_hash_key_change()
Description
This test case verifies changing device’s RSS hash key takes
into affect.
Steps:
1. Skip the test if the kernel version is any less than LTS 5.
2. Get all the device’s RSS hash key values.
3. Swap the last 2 characters of original hash key to make a new hash key.
4. Validate changing the hash key setting using the new hash key.
5. Revert back the settings to original values.
Priority

2

validate_device_rx_hash_level_change()
Description
This test case verifies whether changing device’s RX hash level
for tcp and udp takes into affect.
Steps:
Note: Same steps are used for both TCP and UDP.
1. Get all the device’s RX hash level status.
2. Depending on current setting, change to enabled/disabled.
3. Validate changing the hash level setting.
4. Revert back the settings to original values.
Priority

2

class Sriov
Description
This test suite uses to verify accelerated network functionality.
Platform

Azure, Ready

Area

sriov

Category

functional

sriov_single_vf_connection_validation()
Description
This case verifies module of sriov network interface is loaded and
each synthetic nic is paired with one VF, and check rx statistics of source
and tx statistics of dest increase after send 200 Mb file from source to dest.
Steps,
1. Check VF of synthetic nic is paired.
2. Check module of sriov network device is loaded.
3. Check VF counts listed from lspci is expected.
4. Setup SSH connection between source and dest with key authentication.
5. Ping the dest IP from the source machine to check connectivity.
6. Generate 200Mb file, copy from source to dest.
7. Check rx statistics of source VF and tx statistics of dest VF is increased.
Priority

1

Requirement

network_interface

sriov_ethtool_offload_setting()
Description
This case verify below two kernel patches.
1. hv_netvsc: Sync offloading features to VF NIC
2. hv_netvsc: Allow scatter-gather feature to be tunable
Steps,
1. Change scatter-gather feature on synthetic nic,
verify the the feature status sync to the VF dynamically.
2. Disable and enable sriov,
check the scatter-gather feature status keep consistent in VF.
Priority

2

Requirement

network_interface

sriov_basic_validation()
Description
This case verifies module of sriov network interface is loaded and each
synthetic nic is paired with one VF.
Steps,
1. Check VF of synthetic nic is paired.
2. Check module of sriov network device is loaded.
3. Check VF counts listed from lspci is expected.
Priority

1

Requirement

network_interface

sriov_disable_enable_on_guest_validation()
Description
This case verify VM works well after down the VF nic and up VF nic inside VM.
Steps,
1. Do the basic sriov check.
2. Do network connection test with bring down the VF nic.
3. After copy 200Mb file from source to desc.
4. Check rx statistics of source synthetic nic and tx statistics of dest
synthetic nic is increased.
5. Bring up VF nic.
Priority

2

Requirement

network_interface

sriov_disable_enable_pci_validation()
Description
This case verify VM works well after disable and enable PCI device inside VM.
Steps,
1. Disable sriov PCI device inside the VM.
2. Enable sriov PCI device inside the VM.
3. Do the basic sriov check.
4. Do VF connection test.
Priority

2

Requirement

network_interface

sriov_disable_enable_validation()
Description
This case verify VM works well after disable and enable accelerated network in
network interface through sdk.
Steps,
1. Do the basic sriov check.
2. Set enable_accelerated_networking as False to disable sriov.
3. Set enable_accelerated_networking as True to enable sriov.
4. Do the basic sriov check.
Priority

2

Requirement

supported_platform_type[AZURE, READY]

sriov_single_vf_connection_max_cpu_validation()
Description
This case needs 2 nodes and 64 Vcpus. And it verifies module of sriov network
interface is loaded and each synthetic nic is paired with one VF, and check
rx statistics of source and tx statistics of dest increase after send 200 Mb
file from source to dest.
Steps,
1. Check VF of synthetic nic is paired.
2. Check module of sriov network device is loaded.
3. Check VF counts listed from lspci is expected.
4. Setup SSH connection between source and dest with key authentication.
5. Ping the dest IP from the source machine to check connectivity.
6. Generate 200Mb file, copy from source to dest.
7. Check rx statistics of source VF and tx statistics of dest VF is increased.
Priority

2

Requirement

network_interface

sriov_provision_with_max_nics_validation()
Description
This case verify VM works well when provison with max (8) sriov nics.
Steps,
1. Provision VM with max network interfaces with enabling accelerated network.
2. Do the basic sriov testing.
Priority

2

Requirement

network_interface

sriov_max_vf_connection_max_cpu_validation()
Description
This case needs 2 nodes, 8 nics and 64 Vcpus. And it verifies module of sriov
network interface is loaded and each synthetic nic is paired with one VF, and
check rx statistics of source and tx statistics of dest increase after send 200
Mb file from source to dest.
Steps,
1. Check VF of synthetic nic is paired.
2. Check module of sriov network device is loaded.
3. Check VF counts listed from lspci is expected.
4. Setup SSH connection between source and dest with key authentication.
5. Ping the dest IP from the source machine to check connectivity.
6. Generate 200Mb file, copy from source to dest.
7. Check rx statistics of source VF and tx statistics of dest VF is increased.
Priority

2

Requirement

network_interface

sriov_reload_modules_validation()
Description
This case verify VM works well during remove and load sriov modules.
Steps,
1. Provision VM with max network interfaces with enabling accelerated network.
2. Do the basic sriov testing.
3. Remove sriov module, check network traffic through synthetic nic.
4. Load sriov module, check network traffic through VF.
Priority

2

Requirement

network_interface

sriov_max_vf_connection_validation()
Description
This case needs 2 nodes and 8 nics. And it verifies module of sriov network
interface is loaded and each synthetic nic is paired with one VF, and check
rx statistics of source and tx statistics of dest increase after send 200 Mb
file from source to dest.
Steps,
1. Check VF of synthetic nic is paired.
2. Check module of sriov network device is loaded.
3. Check VF counts listed from lspci is expected.
4. Setup SSH connection between source and dest with key authentication.
5. Ping the dest IP from the source machine to check connectivity.
6. Generate 200Mb file, copy from source to dest.
7. Check rx statistics of source VF and tx statistics of dest VF is increased.
Priority

2

Requirement

network_interface

sriov_add_max_nics_validation()
Description
This case verify VM works well after attached the max sriov nics after
provision.
Steps,
1. Attach 7 extra sriov nic into the VM.
2. Do the basic sriov testing.
Priority

2

Requirement

network_interface

class Synthetic
Description
This test suite uses to verify synthetic network functionality.
Platform

Azure, Ready

Area

network

Category

functional

synthetic_add_max_nics_one_time_after_provision_validation()
Description
This case verify VM works well after attaching 7 extra synthetic nics
in one time.
Steps,
1. Provision VM with 1 network interface with synthetic network.
2. Add 7 extra network interfaces in one time.
3. Check each nic has an ip address.
Priority

2

Requirement

network_interface

synthetic_add_max_nics_one_by_one_after_provision_validation()
Description
This case verify VM works well after attaching 7 extra synthetic nics
one by one.
Steps,
1. Provision VM with 1 network interface with synthetic network.
2. Add 7 extra network interfaces one by one.
3. Check each nic has an ip address.
Priority

2

Requirement

network_interface

synthetic_provision_with_max_nics_validation()
Description
This case verify VM works well when provison with max (8) synthetic nics.
Steps,
1. Provision VM with max network interfaces with synthetic network.
2. Check each nic has an ip address.
Priority

2

Requirement

network_interface

class NetInterface
Description
This test suite validates basic functionalities of Network Interfaces.
Platform

Azure, Ready

Area

network

Category

functional

validate_netvsc_reload()
Description
This test case verifies if synthetic network module - netvsc
can be reloaded gracefully when done multiple times.
Steps:
1. Validate netvsc isn’t built-in already. If it is then skip the test.
2. Unload and load netvsc module multiple times in a loop.
Priority

1

Requirement

network_interface

class Xfstesting
Description
This test suite is to validate different types of data disk on Linux VM
using xfstests.
Platform

Azure, Ready

Area

storage

Category

community

xfstesting_ext4_standard_datadisk_validation()
Description
This test case will run ext4 xfstests testing against
standard data disk with ext4 type system.
Priority

3

Requirement

disk

xfstesting_btrfs_nvme_datadisk_validation()
Description
This test case will run btrfs xfstests testing against
nvme data disk with btrfs type system.
Priority

3

Requirement

supported_features[Nvme]

xfstesting_btrfs_standard_datadisk_validation()
Description
This test case will run btrfs xfstests testing against
standard data disk with btrfs type system.
Priority

3

Requirement

disk

xfstesting_xfs_nvme_datadisk_validation()
Description
This test case will run xfs xfstests testing against
nvme data disk with xfs type system.
Priority

3

Requirement

supported_features[Nvme]

xfstesting_generic_standard_datadisk_validation()
Description
This test case will run generic xfstests testing against
standard data disk with xfs type system.
Priority

3

Requirement

disk

xfstesting_ext4_nvme_datadisk_validation()
Description
This test case will run ext4 xfstests testing against
nvme data disk with ext4 type system.
Priority

3

Requirement

supported_features[Nvme]

xfstesting_xfs_standard_datadisk_validation()
Description
This test case will run xfs xfstests testing against
standard data disk with xfs type system.
Priority

3

Requirement

disk

xfstesting_generic_nvme_datadisk_validation()
Description
This test case will run generic xfstests testing against
nvme data disk with xfs type system.
Priority

3

Requirement

supported_features[Nvme]