Monitor and manage your deployment

Citrix Hypervisor provides detailed monitoring of performance metrics. These metrics include CPU, memory, disk, network, C-state/P-state information, and storage. Where appropriate, these metrics are available on a per host and a per VM basis. These metrics are available directly, or can be accessed and viewed graphically in Citrix Hypervisor Center or other third-party applications.

Citrix Hypervisor also provides system and performance alerts. Alerts are notifications that occur in response to selected system events. These notifications also occur when one of the following values goes over a specified threshold on a managed host, VM, or storage repository: CPU usage, network usage, memory usage, control domain memory usage, storage throughput, or VM disk usage. You can configure the alerts by using the xe CLI or by using Citrix Hypervisor Center. To create notifications based on any of the available Host or VM performance metrics seePerformance alerts．

Monitor Citrix Hypervisor performance

Customers can monitor the performance of their Citrix Hypervisor servers and Virtual Machines (VMs) using the metrics exposed through Round Robin Databases (RRDs). These metrics can be queried over HTTP or through the RRD2CSV tool. In addition, Citrix Hypervisor Center uses this data to produce system performance graphs. For more information, seeAnalyze and visualize metrics．

The following tables list all of the available Host and VM metrics.

Notes:

• Latency over a period is defined as the average latency of operations during that period.
• The availability and utility of certain metrics are SR and CPU dependent.
• Performance metrics are not available for GFS2 SRs and disks on those SRs.

Available host metrics

Available VM metrics

Note:

The value ofcan be any of the following:

NIC	If contains pif_eth#, where ## is 0–9
	If contains pif_eth#.## or pif_xenbr## or pif_bond##
Network	If contains pif_xapi##, (note that appears as is)
TAP	If contains pif_tap##
xapi Loopback	如果<论坛>包含pif_lo

Analyze and visualize metrics

The Performance tab in Citrix Hypervisor Center provides real time monitoring of performance statistics across resource pools in addition to graphical trending of virtual and physical machine performance. Graphs showing CPU, memory, network, and disk I/O are included on the Performance tab by default. You can add more metrics, change the appearance of the existing graphs or create extra ones. For more information, seeConfiguring metricsin the following section.

• You can view up to 12 months of performance data and zoom in to take a closer look at activity spikes.

• Citrix Hypervisor Center can generate performance alerts when CPU, memory, network I/O, storage I/O, or disk I/O usage exceed a specified threshold on a server, VM, or SR. For more information, seeAlertsin the following section.

Note:

Install the Citrix VM Tools to see full VM performance data.

Configure performance graphs

To add a graph:

1. On thePerformancetab, clickActionsand thenNew Graph．The New Graph dialog box is displayed.

2. In theNamefield, enter a name for the graph.

3. From the list ofDatasources, select the check boxes for the datasources you want to include in the graph.

4. ClickSave．

To edit an existing graph:

1. Navigate to thePerformancetab, and select the graph that you would like to modify.

2. Right-click on the graph and selectActions, or click theActionsbutton. Then selectEdit Graph．

3. On the graph details window, make the necessary changes, and clickOK．

Configure the graph type

Data on the performance graphs can be displayed as lines or as areas. To change the graph type:

1. On theToolsmenu, clickOptionsand selectGraphs．

2. To view performance data as a line graph, click theLine graphoption.

3. To view performance data as an area graph, click theArea graphoption.

4. ClickOKto save your changes.

Comprehensive details for configuring and viewing Citrix Hypervisor Center performance graphs can be found in the Citrix Hypervisor Center documentation in the sectionMonitoring System Performance．

Configure metrics

Note:

C-states and P-states are power management features of some processors. The range of states available depends on the physical capabilities of the host, as well power management configuration.

Both host and VM commands return the following:

• A full description of the data source

• The units applied to the metric

• The range of possible values that may be used

For example:

name_label: cpu0-C1 name_description: Proportion of time CPU 0 spent in C-state 1 enabled: true standard: true min: 0.000 max: 1.000 units: Percent 

Enable a specific metric

Most metrics are enabled and collected by default, to enable those metrics that are not, enter the following:

xe host-data-source-record data-source=metric name host=hostname 

Disable a specific metric

You may not want to collect certain metrics regularly. To disable a previously enabled metric, enter the following:

xe host-data-source-forget data-source=metric name host=hostname 

Display a list of currently enabled host metrics

To list the host metrics currently being collected, enter the following:

xe host-data-source-list host=hostname 

Display a list of currently enabled VM metrics

To host the VM metrics currently being collected, enter the following:

xe vm-data-source-list vm=vm_name 

Use RRDs

Citrix Hypervisor uses RRDs to store performance metrics. These RRDs consist of multiple Round Robin Archives (RRAs) in a fixed size database.

Each archive in the database samples its particular metric on a specified granularity:

• Every 5 seconds for 10 minutes
• Every minute for the past two hours
• Every hour for the past week
• Every day for the past year

The sampling that takes place every five seconds records actual data points, however the following RRAs use Consolidation Functions instead. The consolidation functions supported by Citrix Hypervisor are:

• AVERAGE
• MIN
• MAX

RRDs exist for individual VMs (including dom0) and the Citrix Hypervisor server. VM RRDs are stored on the host on which they run, or the pool coordinator when not running. Therefore the location of a VM must be known to retrieve the associated performance data.

For detailed information on how to use Citrix Hypervisor RRDs, see theCitrix Hypervisor Software Development Kit Guide．

Analyze RRDs using HTTP

You can download RRDs over HTTP from the Citrix Hypervisor server specified using the HTTP handler registered at/host_rrdor/vm_rrd．这两个地址需要uthentication either by HTTP authentication, or by providing a valid management API session references as a query argument. For example:

Download a Host RRD.

wget http://server/host_rrd?session_id=OpaqueRef:SESSION HANDLE> 

Download a VM RRD.

wget http://server/vm_rrd?session_id=OpaqueRef:SESSION HANDLE>&uuid=VM UUID> 

Both of these calls download XML in a format that can be imported into therrdtoolfor analysis, or parsed directly.

Analyze RRDs using rrd2csv

In addition to viewing performance metrics in Citrix Hypervisor Center, the rrd2csv tool logs RRDs to Comma Separated Value (CSV) format. Man and help pages are provided. To display the rrd2csv tool man or help pages, run the following command:

man rrd2csv 

Or

rrd2csv --help 

Note:

Where multiple options are used, supply them individually. For example: to return both the UUID and the name-label associated with a VM or a host, call rrd2csv as shown below:

rrd2csv -u -n

The UUID returned is unique and suitable as a primary key, however the name-label of an entity may not necessarily be unique.

The man page (rrd2csv --help) is the definitive help text of the tool.

Alerts

You can configure Citrix Hypervisor to generate alerts based on any of the available Host or VM Metrics. In addition, Citrix Hypervisor provides preconfigured alerts that trigger when hosts undergo certain conditions and states. You can view these alerts using Citrix Hypervisor Center or the xe CLI.

View alerts using Citrix Hypervisor Center

You can view different types of alerts in Citrix Hypervisor Center by clickingNotificationsand thenAlerts．The Alerts view displays various types of alerts, including Performance alerts, System alerts, and Software update alerts.

Performance alerts

Performance alerts can be generated when one of the following values exceeds a specified threshold on a managed host, VM, or storage repository (SR): CPU usage, network usage, memory usage, control domain memory usage, storage throughput, or VM disk usage.

By default, the alert repeat interval is set to 60 minutes, it can be modified if necessary. Alerts are displayed on the Alerts page in the Notifications area in Citrix Hypervisor Center. You can also configure Citrix Hypervisor Center to send an email for any specified performance alerts along with other serious system alerts.

Any customized alerts that are configured using the xe CLI are also displayed on the Alerts page in Citrix Hypervisor Center.

Each alert has a corresponding priority/severity level. You can modify these levels and optionally choose to receive an email when the alert is triggered. The default alert priority/severity is set at3．

Configure performance alerts

1. In theResourcespane, select the relevant host, VM, or SR, then click theGeneraltab and thenProperties．

2. Click theAlertstab. You can configure the following alerts:

   • CPU使用率alerts for a host or VM: Check theGenerate CPUusage alerts check box, then set the CPU usage and time threshold that trigger the alert

   • Network usagealerts for a host or VM: Check theGenerate network usage alertscheck box, then set the network usage and time threshold that trigger the alert.

   • Memory usagealerts for a host: Check the生成内存使用啤酒rtscheck box, and then set the free memory and time threshold that trigger the alert.

   • Control domain memory usagealerts for a host: Check theGenerate control domain memory usage alertscheck box, and then set the control domain memory usage and time threshold that trigger the alert.

   • Disk usagealerts for a VM: Check theGenerate disk usage alertscheck box, then set the disk usage and time threshold that trigger the alert.

   • Storage throughputalerts for an SR: Check theGenerate storage throughput alertscheck box, then set the storage throughput and time threshold that trigger the alert.

     Note:

     Physical Block Devices (PBD) represent the interface between a specific Citrix Hypervisor server and an attached SR. When the total read/write SR throughput activity on a PBD exceeds the threshold you have specified, alerts are generated on the host connected to the PBD. Unlike other Citrix Hypervisor server alerts, this alert must be configured on the SR.

3. To change the alert repeat interval, enter the number of minutes in theAlert repeat intervalbox. When an alert threshold has been reached and an alert generated, another alert is not generated until after the alert repeat interval has elapsed.

4. ClickOKto save your changes.

For comprehensive details on how to view, filter and configure severities for performance alerts, see配置性能警报in the Citrix Hypervisor Center documentation.

System alerts

The following table displays the system events/conditions that trigger an alert to be displayed on the Alerts page in Citrix Hypervisor Center.

Software update alerts

• Citrix Hypervisor Center old:Citrix Hypervisor expects a newer version but can still connect to the current version
• Citrix Hypervisor Center out of date:Citrix Hypervisor Center is too old to connect to Citrix Hypervisor
• Citrix Hypervisor out of date:Citrix Hypervisor is an old version that the current Citrix Hypervisor Center cannot connect to
• License expired alert:Citrix Hypervisor license has expired
• Missing IQN alert:Citrix Hypervisor uses iSCSI storage but the host IQN is blank
• Duplicate IQN alert:Citrix Hypervisor uses iSCSI storage, and there are duplicate host IQNs

Configure performance alerts by using the xe CLI

Note:

Triggers for alerts are checked at a minimum interval of five minutes. This interval avoids placing excessive load on the system to check for these conditions and reporting of false positives. Setting an alert repeat interval smaller than five minutes results in the alerts still being generated at the five minute minimum interval.

The performance monitoringperfmontool runs once every five minutes and requests updates from Citrix Hypervisor which are averages over one minute. These defaults can be changed in/etc/sysconfig/perfmon．

Theperfmontool reads updates every five minutes of performance variables running on the same host. These variables are separated into one group relating to the host itself, and a group for each VM running on that host. For each VM and host,perfmonreads the parameterother-config:perfmon并使用这个字符串,以确定哪些变量to monitor, and under which circumstances to generate a message.

For example, the following shows an example of configuring a VM “CPU usage” alert by writing an XML string into the parameterother-config:perfmon:

xe vm-param-set uuid=vm_uuid other-config:perfmon=\ '     ' 

Note:

You can use multiple variable nodes.

After setting the new configuration, use the following command to refreshperfmonfor each host:

xe host-call-plugin host=host_uuid plugin=perfmon fn=refresh 

如果刷新不是这么做的,有一个延迟更加积极re the new configuration takes effect, since by default,perfmonchecks for new configuration every 30 minutes. This default can be changed in/etc/sysconfig/perfmon．

Valid VM elements

• name: The name of the variable (no default). If the name value is eithercpu_usage,network_usage, ordisk_usage, therrd_regexandalarm_trigger_senseparameters are not required as defaults for these values are used.

• alarm_priority: The priority of the alerts generated (default3).

• alarm_trigger_level: The level of value that triggers an alert (no default).

• alarm_trigger_sense: The value ishighifalarm_trigger_levelis a maximum value otherwiselowif thealarm_trigger_levelis a minimum value (the defaulthigh).

• alarm_trigger_period: The number of seconds that values (above or below the alert threshold) can be received before an alert is sent (the default is60).

• alarm_auto_inhibit_period: The number of seconds this alert will be disabled after an alert is sent (the default is3600).

• consolidation_fn: Combines variables from rrd_updates into one value. Forcpu-usagethe default isaverage, forfs_usagethe default isget_percent_fs_usageand for all others -sum．

• rrd_regex: Matches the names of variables fromxe vm-data-sources-list uuid=vm_uuid, to compute performance values. This parameter has defaults for the named variables:

  • cpu_usage
  • network_usage
  • disk_usage

If specified, the values of all items returned byxe vm-data-source-listwhose names match the specified regular expression are consolidated using the method specified as theconsolidation_fn．

Valid host elements

• name: The name of the variable (no default).
• alarm_priority: The priority of the alerts generated (default3).
• alarm_trigger_level: The level of value that triggers an alert (no default).
• alarm_trigger_sense: The value ishighwhenalarm_trigger_levelis a maximum value otherwiselowif thealarm_trigger_levelis a minimum value. (defaulthigh)
• alarm_trigger_period: The number of seconds that values (above or below the alert threshold) can be received before an alert is sent (default60).
• alarm_auto_inhibit_period: The number of seconds that the alert is disabled for after an alert is sent. (default3600).
• consolidation_fn: Combines variables fromrrd_updatesinto one value (defaultsum- oraverage)
• rrd_regex: A regular expression to match the names of variables returned by thexe vm-data-source-list uuid=vm_uuidcommand to use to compute the statistical value. This parameter has defaults for the following named variables:
  • cpu_usage
  • network_usage
  • memory_free_kib
  • sr_io_throughput_total_xxxxxxxx (wherexxxxxxxxis the first eight characters of the SR-UUID).

SR Throughput: Storage throughput alerts must be configured on the SR rather than the host. For example:

xe sr-param-set uuid=sr_uuid other-config:perfmon=\ '     ' 

Generic example configuration

The following example shows a generic configuration:

            ...  ...  

Configure email alerts

You can configure Citrix Hypervisor to send email notifications when Citrix Hypervisor servers generate alerts. This configuration can be done either by using Citrix Hypervisor Center, or by using the xe Command Line Interface (CLI).

Enable email alerts by using Citrix Hypervisor Center

1. In the Resources pane, right-click on a pool and selectProperties．

2. In the Properties window, selectEmail Options．

3. Select the Send email alert notifications check box and enter the email address and SMTP server details.

   Note:

   Enter the details of an SMTP server which does not require authentication

4. Choose the preferred language from theMail languagelist to receive performance alert email. The three languages available are English, Chinese, and Japanese.

   The default language for configuring performance alert email language for Citrix Hypervisor Center is English.

Enable email alerts by using the xe CLI

Important:

When using Citrix Hypervisor Center or the xe CLI to enable email notifications, enter the details of an SMTP server, which does not require authentication. Emails sent through SMTP servers which require authentication are not delivered.

To configure email alerts, specify the email address and SMTP server:

xe pool-param-set uuid=pool_uuid other-config:mail-destination=joe.bloggs@domain.tld xe pool-param-set uuid=pool_uuid other-config:ssmtp-mailhub=smtp.domain.tld[:port] 

You can also specify the minimum value of the priority (known as ‘severity’ in Citrix Hypervisor Center) field in the message before the email is sent:

xe pool-param-set uuid=pool_uuid other-config:mail-min-priority=level 

The default priority level is3．

Note:

Some SMTP servers only forward mails with addresses that use FQDNs. If you find that emails are not being forwarded it might be for this reason. In which case, you can set the server host name to the FQDN so this address is used when connecting to your mail server.

To configure performance alert mail language:

xe pool-param-set uuid=pool_uuid other-config:mail-language=en-US | zh-CN | ja-JP 

Send email alerts through authenticated SMTP servers

The mail-alarm utility in Citrix Hypervisor uses sSMTP to send email notifications. Before sending email notifications, the mail-alarm utility looks for the configuration file,mail-alarm.conf．If the configuration file exists, the contents of the file are used to configure sSMTP. Otherwise the details available in the XAPI database (as configured using Citrix Hypervisor Center or the xe CLI) are used to send email alerts. To send email notifications through authenticated SMTP servers, create amail-alarm.conffile in/etc/用下面的内容:

root=postmaster authUser= authPass= mailhub=: 

Note:

This configuration file is used for all alerts generated by Citrix Hypervisor servers.

Extra configuration options

Each SMTP server can differ slightly in its setup and may require extra configuration. The following extract from thessmtp.confman page shows the correct syntax and available options:

NAME ssmtp.conf – ssmtp configuration file DESCRIPTION ssmtp reads configuration data from /etc/ssmtp/ssmtp.conf The file con- tains keyword-argument pairs, one per line. Lines starting with '//m.giftsix.com/docs/en-us/citrix-hypervisor/#' and empty lines are interpreted as comments. The possible keywords and their meanings are as follows (both are case- insensitive): Root The user that gets all mail for userids less than 1000. If blank, address rewriting is disabled. Mailhub The host to send mail to, in the form host | IP_addr port [: port]. The default port is 25. RewriteDomain The domain from which mail seems to come. For user authentication. Hostname The full qualified name of the host. If not specified, the host is queried for its hostname. FromLineOverride Specifies whether the From header of an email, if any, may over- ride the default domain. The default is "no". UseTLS Specifies whether ssmtp uses TLS to talk to the SMTP server. The default is "no". UseSTARTTLS Specifies whether ssmtp does a EHLO/STARTTLS before starting TLS negotiation. See RFC 2487. TLSCert The file name of an RSA certificate to use for TLS, if required. AuthUser The user name to use for SMTP AUTH. The default is blank, in which case SMTP AUTH is not used. AuthPass The password to use for SMTP AUTH. AuthMethod The authorization method to use. If unset, plain text is used. May also be set to "cram-md5". 

Custom fields and tags

Citrix Hypervisor Center supports the creation of tags and custom fields, which allows for organization and quick searching of VMs, storage and so on. For more information, seeMonitoring System Performance．

Custom searches

Citrix Hypervisor Center supports the creation of customized searches. Searches can be exported and imported, and the results of a search can be displayed in the navigation pane. For more information, seeMonitoring System Performance．

Determine throughput of physical bus adapters

For FC, SAS and iSCSI HBAs you can determine the network throughput of your PBDs using the following procedure.

1. List the PBDs on a host.
2. Determine which LUNs are routed over which PBDs.
3. For each PBD and SR, list the VBDs that reference VDIs on the SR.
4. For all active VBDs that are attached to VMs on the host, calculate the combined throughput.

For iSCSI and NFS storage, check your network statistics to determine if there is a throughput bottleneck at the array, or whether the PBD is saturated.