ESXI tips

修改WEB访问的端口

默认https的端口是443,某些情况下,443有可能被禁了,所以想用其他端口,此时可以直接在下面文件中修改端口,重启即可:

[root@localhost:~] more /etc/vmware/rhttpproxy/config.xml | grep 443
      <httpsPort>443</httpsPort>

通过命令行启动虚机

重启ESXI主机后,如果发现重要的虚拟机忘记放入开启自启动了,此时还无法通过web页面访问ESXI,但SSH可以访问,此时就需要通过命令行来启动虚拟机,方法如下:

[root@localhost:~] vim-cmd vmsvc/getallvms
Vmid           Name                                    File                                 Guest OS         Version             Annotation           
10     xrv9k-731-1             [xxx] xrv9k-731-1/xrv9k-731-1.vmx                       otherGuest64          vmx-09                                   
12     xrv9k-731-2             [xxx] xrv9k-731-2/xrv9k-731-1.vmx                       otherGuest64          vmx-09                                   
14     VMware vCenter Server   [xxx] VMware vCenter Server/VMware vCenter Server.vmx   other3xLinux64Guest   vmx-10    VMware vCenter Server Appliance
2      centos7.9-beihe         [xxx] centos7.9/centos7.9.vmx                           centos7_64Guest       vmx-18                                   
3      centos7.9-qiufan        [xxx] centos7.9-xxxxxx/centos7.9.vmx                    centos7_64Guest       vmx-18                                   
4      centos7.9-songjian      [xxx] centos7.9-xxxxxxxx/centos7.9.vmx                  centos7_64Guest       vmx-18                                   
6      win10                   [xxx] win10/win10.vmx                                   windows9_64Guest      vmx-18                                   
8      EVE-C-112               [xxx] EVE-C-112/EVE-C-112.vmx                           ubuntu64Guest         vmx-11                                   
9      centos7-yangsuit        [xxx] centos7-yangsuit/centos7-yangsuit.vmx             centos7_64Guest       vmx-18                                   
[root@localhost:~] vim-cmd vmsvc/power.on 6
Powering on VM:

远程更改ESXI的管理IP

机房变更,需要更改ESXI的管理IP地址,但又不方便去现场连显示器和键盘,是否可以远程更改IP地址?答案是可以的,只要你能ssh到ESXI的shell里:

[root@localhost:~] TERM=xterm
[root@localhost:~] dcui

如果你发现输入f2没有反应,那么在MacOS里的键盘设置中直接勾选下面项目即可:

离线升级7.0.0 to 7.0.3

先下载升级包:VMware-ESXi-7.0U3-18644231-depot.zip

然后通过Web上传到存储空间中,通过SSH进入,执行下面命令(注意-d后面要写绝对路径,否则会报错,即是你在image目录下,也要输入绝对路径):

[root@localhost:~]  esxcli software sources profile list -d /vmfs/volumes/datastore1/image/VMware-ESXi-7.0U3-18644231-depot.zip 
Name                          Vendor        Acceptance Level  Creation Time        Modification Time
----------------------------  ------------  ----------------  -------------------  -----------------
ESXi-7.0U3-18644231-standard  VMware, Inc.  PartnerSupported  2021-10-05T00:00:00  2021-10-05T00:00:00
ESXi-7.0U3-18644231-no-tools  VMware, Inc.  PartnerSupported  2021-10-05T00:00:00  2021-09-17T16:22:28
[root@localhost:~] 

然后用下面命令升级,注意/tmp目录下的空间十分有限,如果遇到下面告警,就近到tmp目录下,删掉里面的所有内容:

 Error unlocking file /tmp/_tmp_stagebootbank.lck: [Errno 28] No space left on device

然后执行升级,重启后完成升级:

[root@localhost:~] esxcli  software  profile update -d /vmfs/volumes/datastore1/image/VMware-ESXi-7.0U3-18644231-depot.zip -p ESXi-7.0U3-18644231-standard
Update Result
   Message: The update completed successfully, but the system needs to be rebooted for the changes to be effective.
   Reboot Required: true
   VIBs Installed: VMW_bootbank_atlantic_1.0.3.0-8vmw.703.0.0.18644231......

2022-01-16 更新:安装VIB固件更新

有时候需要安装第三方固件更新一些工具包,如Mellanox的工具包,但安装时会有下面报错:

[root@localhost:~] esxcli software vib install -v /tmp/mft-4.18.0.107-10EM-700.0.0.15843807.x86_64.vib 
 [InstallationError]
 ('MEL_bootbank_mft_4.18.0.107-0', 'Could not find a trusted signer: certificate has expired')
      cause = ('MEL_bootbank_mft_4.18.0.107-0', 'Could not find a trusted signer: certificate has expired')
       vibs = ['MEL_bootbank_mft_4.18.0.107-0']
 Please refer to the log file for more details.

这时可以通过下面方法不去确认证书:

[root@localhost:~] esxcli software acceptance set --level=CommunitySupported
Host acceptance level changed to 'CommunitySupported'.
[root@localhost:~] esxcli software vib install -v /tmp/mft-4.18.0.107-10EM-700.0.0.15843807.x86_64.vib --no-sig-check
Installation Result
   Message: The update completed successfully, but the system needs to be rebooted for the changes to be effective.
   Reboot Required: true
   VIBs Installed: MEL_bootbank_mft_4.18.0.107-0
   VIBs Removed: 
   VIBs Skipped: 
[root@localhost:~] esxcli software acceptance set --level=PartnerSupported
Host acceptance level changed to 'PartnerSupported'.

更改vCenter7的IP地址

注意:这部分是更改IP的操作,但更改IP后,发现无法访问,所以删了重装(重装过程看下一个章节);重装后发现遗漏了一个重要步骤,那就是需要更改下/etc/hosts中的域名。。。

直接通过Console接口进去后,更改IP总是报下面错误:

经过查找,需要通过Alt + F1(在MacOS,Option+F1)进入shell,然后执行下面命令通过命令行修改IPv4地址、网关和DNS:

/opt/vmware/share/vami/vami_config_net

无DNS部署vCenter7.0.3

首先完成第一阶段的安装,配置如下,FQDN和DNS都设置成自己:

直接登录vCenter,需要通过Alt + F1(在MacOS,Option+F1)进入shell,修改/etc/hosts,加入下面一行

10.114.252.142    localhost

最后通过“https://10.114.252.142:5480”登录vCenter并完成第二阶段的安装,配置如下:

这部分参考网站:无DNS解析环境下部署VCenter(VCSA)7.0

2022-01-17 更新:在ESXI开启SR-IOV

最近在测试虚拟测试仪的性能,需要使用SR-IOV,PCI Passthrough跟SR-IOV有什么区别呢?下面是从网上趴下来的图片,看后会很清晰,简单理解就是一个是独享,一个是共享;

网卡用的是Mellanox CX-5单口100g,需要单独开启SR-IOV,详细可以看这里,下面是简单总结:

安装Mellanox工具包

下载mft并安装,具体参考本文中的:“2022-01-16 更新:安装VIB固件更新”;

查看网卡的默认状态

[root@localhost:~] /opt/mellanox/bin/mst status
MST devices:
------------
mt4119_pciconf0              
mt4119_pciconf1              
mt4125_pciconf2              
mt4119_pciconf3              
[root@localhost:~] /opt/mellanox/bin/mlxconfig -d mt4119_pciconf1 q

Device #1:
----------

Device type:    ConnectX5       
Name:           MCX515A-CCA_Ax  
Description:    ConnectX-5 EN network interface card; 100GbE single-port QSFP28; PCIe3.0 x16; tall bracket; ROHS R6
Device:         mt4119_pciconf1 

Configurations:                              Next Boot
         MEMIC_BAR_SIZE                      0               
         MEMIC_SIZE_LIMIT                    _256KB(1)       
         HOST_CHAINING_MODE                  DISABLED(0)     
         HOST_CHAINING_DESCRIPTORS           Array[0..7]     
         HOST_CHAINING_TOTAL_BUFFER_SIZE     Array[0..7]     
         FLEX_PARSER_PROFILE_ENABLE          0               
         FLEX_IPV4_OVER_VXLAN_PORT           0               
         ROCE_NEXT_PROTOCOL                  254             
         ESWITCH_HAIRPIN_DESCRIPTORS         Array[0..7]     
         ESWITCH_HAIRPIN_TOT_BUFFER_SIZE     Array[0..7]     
         PF_BAR2_SIZE                        0               
         NON_PREFETCHABLE_PF_BAR             False(0)        
         VF_VPD_ENABLE                       False(0)        
         STRICT_VF_MSIX_NUM                  False(0)        
         VF_NODNIC_ENABLE                    False(0)        
         NUM_PF_MSIX_VALID                   True(1)         
         NUM_OF_VFS                          0               
         NUM_OF_PF                           1               
         PF_BAR2_ENABLE                      False(0)        
         SRIOV_EN                            True(1)         
         PF_LOG_BAR_SIZE                     5               
         VF_LOG_BAR_SIZE                     1               
         NUM_PF_MSIX                         63              
         NUM_VF_MSIX                         19              
         INT_LOG_MAX_PAYLOAD_SIZE            AUTOMATIC(0)    
         PCIE_CREDIT_TOKEN_TIMEOUT           0               
         PARTIAL_RESET_EN                    False(0)        
         SW_RECOVERY_ON_ERRORS               False(0)        
         RESET_WITH_HOST_ON_ERRORS           False(0)        
         ADVANCED_POWER_SETTINGS             False(0)        
         CQE_COMPRESSION                     BALANCED(0)     
         IP_OVER_VXLAN_EN                    False(0)        
         MKEY_BY_NAME                        False(0)        
         ESWITCH_IPV4_TTL_MODIFY_ENABLE      False(0)        
         PRIO_TAG_REQUIRED_EN                False(0)        
         UCTX_EN                             True(1)         
         PCI_ATOMIC_MODE                     PCI_ATOMIC_DISABLED_EXT_ATOMIC_ENABLED(0)
         LRO_LOG_TIMEOUT0                    6               
         LRO_LOG_TIMEOUT1                    7               
         LRO_LOG_TIMEOUT2                    8               
         LRO_LOG_TIMEOUT3                    13              
         LOG_DCR_HASH_TABLE_SIZE             11              
         DCR_LIFO_SIZE                       16384           
         ROCE_CC_PRIO_MASK_P1                255             
         CLAMP_TGT_RATE_AFTER_TIME_INC_P1    True(1)         
         CLAMP_TGT_RATE_P1                   False(0)        
         RPG_TIME_RESET_P1                   300             
         RPG_BYTE_RESET_P1                   32767           
         RPG_THRESHOLD_P1                    1               
         RPG_MAX_RATE_P1                     0               
         RPG_AI_RATE_P1                      5               
         RPG_HAI_RATE_P1                     50              
         RPG_GD_P1                           11              
         RPG_MIN_DEC_FAC_P1                  50              
         RPG_MIN_RATE_P1                     1               
         RATE_TO_SET_ON_FIRST_CNP_P1         0               
         DCE_TCP_G_P1                        1019            
         DCE_TCP_RTT_P1                      1               
         RATE_REDUCE_MONITOR_PERIOD_P1       4               
         INITIAL_ALPHA_VALUE_P1              1023            
         MIN_TIME_BETWEEN_CNPS_P1            0               
         CNP_802P_PRIO_P1                    6               
         CNP_DSCP_P1                         48              
         LLDP_NB_DCBX_P1                     False(0)        
         LLDP_NB_RX_MODE_P1                  OFF(0)          
         LLDP_NB_TX_MODE_P1                  OFF(0)          
         DCBX_IEEE_P1                        True(1)         
         DCBX_CEE_P1                         True(1)         
         DCBX_WILLING_P1                     True(1)         
         KEEP_ETH_LINK_UP_P1                 True(1)         
         KEEP_IB_LINK_UP_P1                  False(0)        
         KEEP_LINK_UP_ON_BOOT_P1             False(0)        
         KEEP_LINK_UP_ON_STANDBY_P1          False(0)        
         NUM_OF_VL_P1                        _4_VLs(3)       
         NUM_OF_TC_P1                        _8_TCs(0)       
         NUM_OF_PFC_P1                       8               
         DUP_MAC_ACTION_P1                   LAST_CFG(0)     
         SRIOV_IB_ROUTING_MODE_P1            LID(1)          
         IB_ROUTING_MODE_P1                  LID(1)          
         PCI_WR_ORDERING                     per_mkey(0)     
         MULTI_PORT_VHCA_EN                  False(0)        
         PORT_OWNER                          True(1)         
         ALLOW_RD_COUNTERS                   True(1)         
         RENEG_ON_CHANGE                     True(1)         
         TRACER_ENABLE                       True(1)         
         IP_VER                              IPv4(0)         
         BOOT_UNDI_NETWORK_WAIT              0               
         UEFI_HII_EN                         False(0)        
         BOOT_DBG_LOG                        False(0)        
         UEFI_LOGS                           DISABLED(0)     
         BOOT_VLAN                           1               
         LEGACY_BOOT_PROTOCOL                PXE(1)          
         BOOT_LACP_DIS                       True(1)         
         BOOT_VLAN_EN                        False(0)        
         BOOT_PKEY                           0               
         ATS_ENABLED                         False(0)        
         DYNAMIC_VF_MSIX_TABLE               False(0)        
         EXP_ROM_UEFI_x86_ENABLE             False(0)        
         EXP_ROM_PXE_ENABLE                  True(1)         
         ADVANCED_PCI_SETTINGS               False(0)        
         SAFE_MODE_THRESHOLD                 10              
         SAFE_MODE_ENABLE                    True(1)  
[root@localhost:~] /opt/mellanox/bin/mlxconfig -d mt4119_pciconf3 q

Device #1:
----------

Device type:    ConnectX5       
Name:           MCX515A-CCA_Ax  
Description:    ConnectX-5 EN network interface card; 100GbE single-port QSFP28; PCIe3.0 x16; tall bracket; ROHS R6
Device:         mt4119_pciconf3 

Configurations:                              Next Boot
......       
         NUM_OF_VFS                          0               
         NUM_OF_PF                           1               

更改网卡驱动并激活SR-IOV

[root@localhost:~] /opt/mellanox/bin/mlxconfig -d mt4119_pciconf1 s SRIOV_EN=1 NUM_OF_VFS=4

Device #1:
----------

Device type:    ConnectX5       
Name:           MCX515A-CCA_Ax  
Description:    ConnectX-5 EN network interface card; 100GbE single-port QSFP28; PCIe3.0 x16; tall bracket; ROHS R6
Device:         mt4119_pciconf1 

Configurations:                              Next Boot       New
         SRIOV_EN                            True(1)         True(1)         
         NUM_OF_VFS                          0               4               

 Apply new Configuration? (y/n) [n] : y
Applying... Done!
-I- Please reboot machine to load new configurations.
[root@localhost:~] 
[root@localhost:~] /opt/mellanox/bin/mlxconfig -d mt4119_pciconf3 s SRIOV_EN=1 NUM_OF_VFS=4

Device #1:
----------

Device type:    ConnectX5       
Name:           MCX515A-CCA_Ax  
Description:    ConnectX-5 EN network interface card; 100GbE single-port QSFP28; PCIe3.0 x16; tall bracket; ROHS R6
Device:         mt4119_pciconf3 

Configurations:                              Next Boot       New
         SRIOV_EN                            True(1)         True(1)         
         NUM_OF_VFS                          0               4               

 Apply new Configuration? (y/n) [n] : y
Applying... Done!
-I- Please reboot machine to load new configurations.

重启ESXI,可以看到已经改了,但可以看到此时没有VF:

[root@localhost:~] /opt/mellanox/bin/mlxconfig -d mt4119_pciconf1 q |grep NUM_OF_VFS
         NUM_OF_VFS                          4               
[root@localhost:~] /opt/mellanox/bin/mlxconfig -d mt4119_pciconf3 q |grep NUM_OF_VFS
         NUM_OF_VFS 
[root@localhost:~] lspci -d | grep -E "0000:23|0000:81"
0000:23:00.0 Ethernet controller: Mellanox Technologies ConnectX-5 EN network interface card 100GbE single-port (MCX515A-CCAT) [vmnic2]
0000:81:00.0 Ethernet controller: Mellanox Technologies ConnectX-5 EN network interface card 100GbE single-port (MCX515A-CCAT) [vmnic6]

打开SR-IOV,可以通过Mellanox工具打开,这时也可以直接在ESXI打开,如下:

[root@localhost:~] lspci -d | grep -E "0000:23|0000:81"
0000:23:00.0 Ethernet controller: Mellanox Technologies ConnectX-5 EN network interface card 100GbE single-port (MCX515A-CCAT) [vmnic2]
0000:23:00.1 Ethernet controller: Mellanox Technologies MT27800 Family [ConnectX-5 Virtual Function] [PF_0.35.0_VF_0]
0000:23:00.2 Ethernet controller: Mellanox Technologies MT27800 Family [ConnectX-5 Virtual Function] [PF_0.35.0_VF_1]
0000:23:00.3 Ethernet controller: Mellanox Technologies MT27800 Family [ConnectX-5 Virtual Function] [PF_0.35.0_VF_2]
0000:23:00.4 Ethernet controller: Mellanox Technologies MT27800 Family [ConnectX-5 Virtual Function] [PF_0.35.0_VF_3]
0000:81:00.0 Ethernet controller: Mellanox Technologies ConnectX-5 EN network interface card 100GbE single-port (MCX515A-CCAT) [vmnic6]
0000:81:00.1 Ethernet controller: Mellanox Technologies MT27800 Family [ConnectX-5 Virtual Function] [PF_0.129.0_VF_0]
0000:81:00.2 Ethernet controller: Mellanox Technologies MT27800 Family [ConnectX-5 Virtual Function] [PF_0.129.0_VF_1]
0000:81:00.3 Ethernet controller: Mellanox Technologies MT27800 Family [ConnectX-5 Virtual Function] [PF_0.129.0_VF_2]
0000:81:00.4 Ethernet controller: Mellanox Technologies MT27800 Family [ConnectX-5 Virtual Function] [PF_0.129.0_VF_3]

注意,如果是1块网卡的2个口,那么用前面的Address 就无法区分了,而ESXI的Web中只能看Address,所以只能进Shell通过lspci来区分,如下所示:

[root@localhost:~] lspci |grep Mellanox
0000:98:00.0 Ethernet controller: Mellanox Technologies ConnectX-6 Dx EN NIC; OCP3.0; 100GbE; dual-port QSFP56; PCIe4.0 x16; (MCX623436AN-CDA) [vmnic0]
0000:98:00.1 Ethernet controller: Mellanox Technologies ConnectX-6 Dx EN NIC; OCP3.0; 100GbE; dual-port QSFP56; PCIe4.0 x16; (MCX623436AN-CDA) [vmnic1]
0000:98:00.2 Ethernet controller: Mellanox Technologies ConnectX Family nmlx5Gen Virtual Function [PF_0.152.0_VF_0]
0000:98:00.3 Ethernet controller: Mellanox Technologies ConnectX Family nmlx5Gen Virtual Function [PF_0.152.0_VF_1]
0000:98:00.4 Ethernet controller: Mellanox Technologies ConnectX Family nmlx5Gen Virtual Function [PF_0.152.0_VF_2]
0000:98:00.5 Ethernet controller: Mellanox Technologies ConnectX Family nmlx5Gen Virtual Function [PF_0.152.0_VF_3]
0000:98:00.6 Ethernet controller: Mellanox Technologies ConnectX Family nmlx5Gen Virtual Function [PF_0.152.0_VF_4]
0000:98:00.7 Ethernet controller: Mellanox Technologies ConnectX Family nmlx5Gen Virtual Function [PF_0.152.0_VF_5]
0000:98:01.0 Ethernet controller: Mellanox Technologies ConnectX Family nmlx5Gen Virtual Function [PF_0.152.0_VF_6]
0000:98:01.1 Ethernet controller: Mellanox Technologies ConnectX Family nmlx5Gen Virtual Function [PF_0.152.0_VF_7]
0000:98:01.2 Ethernet controller: Mellanox Technologies ConnectX Family nmlx5Gen Virtual Function [PF_0.152.1_VF_0]
0000:98:01.3 Ethernet controller: Mellanox Technologies ConnectX Family nmlx5Gen Virtual Function [PF_0.152.1_VF_1]
0000:98:01.4 Ethernet controller: Mellanox Technologies ConnectX Family nmlx5Gen Virtual Function [PF_0.152.1_VF_2]
0000:98:01.5 Ethernet controller: Mellanox Technologies ConnectX Family nmlx5Gen Virtual Function [PF_0.152.1_VF_3]
0000:98:01.6 Ethernet controller: Mellanox Technologies ConnectX Family nmlx5Gen Virtual Function [PF_0.152.1_VF_4]
0000:98:01.7 Ethernet controller: Mellanox Technologies ConnectX Family nmlx5Gen Virtual Function [PF_0.152.1_VF_5]
0000:98:02.0 Ethernet controller: Mellanox Technologies ConnectX Family nmlx5Gen Virtual Function [PF_0.152.1_VF_6]
0000:98:02.1 Ethernet controller: Mellanox Technologies ConnectX Family nmlx5Gen Virtual Function [PF_0.152.1_VF_7]

2022-01-21 更新:Disable Flow control on ESXI Server

ESXI默认是开启Flow Control的,用下面命令查看并关闭Flow Control,详细可以看官方文档

[root@localhost:~] esxcli network nic pauseParams list
NIC     Pause Params Supported  Pause RX  Pause TX  Auto Negotiation  Auto Negotiation Resolution Avail  RX Auto Negotiation Resolution  TX Auto Negotiation Resolution
------  ----------------------  --------  --------  ----------------  ---------------------------------  ------------------------------  ------------------------------
vmnic0                    true      true      true             false                              false                           false                           false
vmnic1                    true      true      true             false                              false                           false                           false
vmnic2                    true      true      true             false                              false                           false                           false
vmnic4                    true      true      true             false                              false                           false                           false
vmnic5                    true      true      true             false                              false                           false                           false
vmnic6                    true      true      true             false                              false                           false                           false
vmnic8                    true     false     false              true                              false                           false                           false
vmnic9                    true      true      true              true                              false                           false                           false
[root@localhost:~] 
[root@localhost:~] esxcli network nic pauseParams set 
Error: Missing required parameter -n|--nic-name

Usage: esxcli network nic pauseParams set [cmd options]

Description: 
  set                   Set pause parameters for a NIC

Cmd options:
  -a|--auto=<bool>      Enable/disable auto negotiation.
  -n|--nic-name=<str>   Name of NIC whose pause parameters should be set. (required)
  -r|--rx=<bool>        Enable/disable pause RX flow control.
  -t|--tx=<bool>        Enable/disable pause TX flow control.
[root@localhost:~] 
[root@localhost:~] esxcli network nic pauseParams set -n vmnic4 -r f
[root@localhost:~] esxcli network nic pauseParams set -n vmnic5 -r f
[root@localhost:~] esxcli network nic pauseParams set -n vmnic4 -t f
[root@localhost:~] esxcli network nic pauseParams set -n vmnic5 -t f
[root@localhost:~]
[root@localhost:~] esxcli network nic pauseParams list
NIC     Pause Params Supported  Pause RX  Pause TX  Auto Negotiation  Auto Negotiation Resolution Avail  RX Auto Negotiation Resolution  TX Auto Negotiation Resolution
------  ----------------------  --------  --------  ----------------  ---------------------------------  ------------------------------  ------------------------------
vmnic0                    true      true      true             false                              false                           false                           false
vmnic1                    true      true      true             false                              false                           false                           false
vmnic2                    true      true      true             false                              false                           false                           false
vmnic4                    true     false     false             false                              false                           false                           false
vmnic5                    true     false     false             false                              false                           false                           false
vmnic6                    true      true      true             false                              false                           false                           false
vmnic8                    true     false     false              true                              false                           false                           false
vmnic9                    true      true      true              true                              false                           false                           false
本文出自 Frank's Blog

版权声明:


本文链接:ESXI tips
版权声明:本文为原创文章,仅代表个人观点,版权归 Frank Zhao 所有,转载时请注明本文出处及文章链接
你可以留言,或者trackback 从你的网站

留言哦

blonde teen swallows load.xxx videos