C11-OpenStack 運算服務:Nova 與 Placement

0. 環境與準備

0.1 本課環境

主機角色管理網 IP
jmp001學員操作機(OpenStack CLI)10.127.0.3
ctl001 / ctl002 / ctl003控制節點(nova_api、nova_scheduler、nova_conductor、placement_api)10.127.0.4 / .14 / .15
scp001運算節點(nova_compute、nova_libvirt),8 核 / 16 GB10.127.0.7
scp002運算節點(nova_compute、nova_libvirt),8 核 / 64 GB10.127.0.18
—VIP10.127.0.250

0.2 連線方式

全班與講師共用 jmp001 與控制節點上的 ubuntu 帳號(具 sudo 權限)。先登入 jmp001,再從 jmp001 SSH 到控制節點:

指令
ssh ubuntu@10.127.0.4

指令標示:[jmp001] 在 jmp001 執行;[ctl] 在任一台控制節點執行。

⚠️ 規則

  • 你使用的是和講師相同的管理員權限,只操作名稱有你編號的資源
  • 預期輸出是示意,UUID、數字、時間、順序可能與你看到的不同

0.3 準備

指令
# [jmp001]
export NN=01                                     # ← 換成你的學員編號(01–20)
source ~/venv/bin/activate
source ~/creds/stu${NN}-openrc.sh
export OS_PLACEMENT_API_VERSION=1.39
echo "c11-stu${NN}"

預期輸出:

預期輸出
c11-stu01

Part A:建立前先看資源帳本(p.11、13、21)

A1. 哪些主機可以當排程候選?

指令
# [jmp001]
openstack compute service list --service nova-compute
openstack hypervisor list

預期輸出:

預期輸出
+--------------------------------------+--------------+---------------------+------+---------+-------+----------------------------+
| ID                                   | Binary       | Host                | Zone | Status  | State | Updated At                 |
+--------------------------------------+--------------+---------------------+------+---------+-------+----------------------------+
| 3629a1fb-a83b-4293-ba41-983b526ad7b0 | nova-compute | scp001-6161072-iaas | nova | enabled | up    | 2026-10-31T15:48:29.000000 |
| 68ec02e1-891a-49d9-a748-dd497dba1f56 | nova-compute | scp002-6164930-iaas | nova | enabled | up    | 2026-10-31T15:48:29.000000 |
+--------------------------------------+--------------+---------------------+------+---------+-------+----------------------------+
+--------------------------------------+---------------------+-----------------+-------------+-------+
| ID                                   | Hypervisor Hostname | Hypervisor Type | Host IP     | State |
+--------------------------------------+---------------------+-----------------+-------------+-------+
| e453b8ae-34de-5374-8da3-b474f2037cb8 | scp001-6161072-iaas | QEMU            | 10.127.0.7  | up    |
| fbc35ee8-6174-5e61-b091-5037b468796b | scp002-6164930-iaas | QEMU            | 10.127.0.18 | up    |
+--------------------------------------+---------------------+-----------------+-------------+-------+

A2. Resource Provider 與 Inventory

每一台 compute node 在 Placement 中就是一個 Resource Provider(RP)。

指令
# [jmp001]
openstack resource provider list

預期輸出:

預期輸出
+--------------------------------------+---------------------+------------+--------------------------------------+----------------------+
| uuid                                 | name                | generation | root_provider_uuid                   | parent_provider_uuid |
+--------------------------------------+---------------------+------------+--------------------------------------+----------------------+
| e453b8ae-34de-5374-8da3-b474f2037cb8 | scp001-6161072-iaas |          8 | e453b8ae-34de-5374-8da3-b474f2037cb8 | None                 |
| fbc35ee8-6174-5e61-b091-5037b468796b | scp002-6164930-iaas |         27 | fbc35ee8-6174-5e61-b091-5037b468796b | None                 |
+--------------------------------------+---------------------+------------+--------------------------------------+----------------------+

把兩台的 uuid 存成變數,後面會一直用到:

指令
# [jmp001]
RP1=$(openstack resource provider list --name scp001-6161072-iaas -f value -c uuid)
RP2=$(openstack resource provider list --name scp002-6164930-iaas -f value -c uuid)
openstack resource provider inventory list $RP1
openstack resource provider inventory list $RP2

預期輸出:

預期輸出
+----------------+------------------+----------+----------+----------+-----------+-------+------+
| resource_class | allocation_ratio | min_unit | max_unit | reserved | step_size | total | used |
+----------------+------------------+----------+----------+----------+-----------+-------+------+
| VCPU           |              4.0 |        1 |        8 |        0 |         1 |     8 |    0 |
| MEMORY_MB      |              1.0 |        1 |    15991 |      512 |         1 | 15991 |    0 |
| DISK_GB        |              1.0 |        1 |      187 |        0 |         1 |   187 |    0 |
+----------------+------------------+----------+----------+----------+-----------+-------+------+
+----------------+------------------+----------+----------+----------+-----------+-------+------+
| resource_class | allocation_ratio | min_unit | max_unit | reserved | step_size | total | used |
+----------------+------------------+----------+----------+----------+-----------+-------+------+
| VCPU           |              4.0 |        1 |        8 |        0 |         1 |     8 |    2 |
| MEMORY_MB      |              1.0 |        1 |    64303 |      512 |         1 | 64303 | 1536 |
| DISK_GB        |              1.0 |        1 |      187 |        0 |         1 |   187 |    6 |
+----------------+------------------+----------+----------+----------+-----------+-------+------+

套用 p.18 的公式 可分配容量 = (total − reserved) × allocation_ratio − 已使用量,填寫下表:

資源scp001 可分配總量scp002 可分配總量
VCPU(8 − 0) × 4.0 = ____(8 − 0) × 4.0 = ____
MEMORY_MB(15991 − 512) × 1.0 = ____(64303 − 512) × 1.0 = ____
DISK_GB________

A3. Trait 與目前用量

指令
# [jmp001]
openstack resource provider trait list $RP1 -f value | wc -l
openstack resource provider trait list $RP1 | grep AVX2
openstack resource provider usage show $RP1
openstack resource provider usage show $RP2

預期輸出:

預期輸出
79
| HW_CPU_X86_AVX2                       |
+----------------+-------+
| resource_class | usage |
+----------------+-------+
| VCPU           |     0 |
| MEMORY_MB      |     0 |
| DISK_GB        |     0 |
+----------------+-------+
+----------------+-------+
| resource_class | usage |
+----------------+-------+
| VCPU           |     2 |
| MEMORY_MB      |  1536 |
| DISK_GB        |     6 |
+----------------+-------+
VCPUMEMORY_MBDISK_GB
scp001(建立前)
scp002(建立前)

Part B:模擬排程:哪裡放得下?(p.15、24、28、30)

allocation candidate list 就是 nova-scheduler 在建立 VM 時向 Placement 問的問題(p.27)。這裡直接手動問,不會建立任何東西。

B1. 一台 m1.tiny 放得下嗎?

指令
# [jmp001]
openstack flavor show m1.tiny -c vcpus -c ram -c disk
openstack allocation candidate list --resource VCPU=1 --resource MEMORY_MB=512 --resource DISK_GB=1 \
  -c "resource provider" -c "inventory used/capacity"

預期輸出:

預期輸出
+-------+-------+
| Field | Value |
+-------+-------+
| disk  | 1     |
| ram   | 512   |
| vcpus | 1     |
+-------+-------+
+--------------------------------------+----------------------------------------------+
| resource provider                    | inventory used/capacity                      |
+--------------------------------------+----------------------------------------------+
| fbc35ee8-6174-5e61-b091-5037b468796b | VCPU=2/32,MEMORY_MB=1536/63791,DISK_GB=6/187 |
| e453b8ae-34de-5374-8da3-b474f2037cb8 | VCPU=0/32,MEMORY_MB=0/15479,DISK_GB=0/187    |
+--------------------------------------+----------------------------------------------+

B2. 三個放不下的例子

指令
# [jmp001]
openstack allocation candidate list --resource VCPU=1 --resource MEMORY_MB=20000 \
  -c "resource provider" -c "inventory used/capacity"

預期輸出:

預期輸出
+--------------------------------------+----------------------------------------------+
| resource provider                    | inventory used/capacity                      |
+--------------------------------------+----------------------------------------------+
| fbc35ee8-6174-5e61-b091-5037b468796b | VCPU=2/32,MEMORY_MB=1536/63791,DISK_GB=6/187 |
+--------------------------------------+----------------------------------------------+

要 20000 MB 記憶體時只剩 scp002:scp001 只有 15479 MB 可分配。兩台加起來雖然有將近 80 GB,但一台 VM 的資源必須來自同一個 RP,這就是 p.30 碎片化問題的根源。

指令
# [jmp001]
openstack allocation candidate list --resource VCPU=9 --resource MEMORY_MB=512

預期輸出:(空白,沒有任何候選)

預期輸出

兩台各有 32 個可分配 VCPU,但 max_unit = 8,單一 VM 要 9 個就沒有候選。實際建立這種 VM 會得到 NoValidHost。

指令
# [jmp001]
openstack allocation candidate list --resource VCPU=1 --resource MEMORY_MB=512 --required CUSTOM_FAST_SSD

預期輸出:

預期輸出
No such trait(s): CUSTOM_FAST_SSD. (HTTP 400)

p.15 的例子要求 CUSTOM_FAST_SSD,但這個 trait 在本環境根本不存在,Placement 直接回 400。換成每台都有的 trait:

指令
# [jmp001]
openstack allocation candidate list --resource VCPU=1 --resource MEMORY_MB=512 --required HW_CPU_X86_AVX2 \
  -c "resource provider" -c "inventory used/capacity"

預期輸出:

預期輸出
+--------------------------------------+----------------------------------------------+
| resource provider                    | inventory used/capacity                      |
+--------------------------------------+----------------------------------------------+
| fbc35ee8-6174-5e61-b091-5037b468796b | VCPU=2/32,MEMORY_MB=1536/63791,DISK_GB=6/187 |
| e453b8ae-34de-5374-8da3-b474f2037cb8 | VCPU=0/32,MEMORY_MB=0/15479,DISK_GB=0/187    |
+--------------------------------------+----------------------------------------------+
查詢結果原因
VCPU=1、MEMORY_MB=512、DISK_GB=12 台兩台都放得下
MEMORY_MB=20000只有 scp002資源必須來自單一 RP
VCPU=90 台超過 max_unit
--required CUSTOM_FAST_SSDHTTP 400trait 不存在
--required HW_CPU_X86_AVX22 台兩台都有這個 trait

Part C:完整建立一台 VM(p.5、12、26)

依照 p.5 的四項啟動要素,用 Boot from Volume 建立一台 VM:

要素本實驗使用由誰提供
Flavorm1.tiny(1 vCPU / 512 MB)Nova、Placement
Source由 cirros 映像建立的 1 GB 開機 volumeGlance、Cinder
Networkdemo-net + default security groupNeutron
Key Pairc11-stuNN-keyNova

C1. 建立 Key Pair

指令
# [jmp001]
openstack keypair create c11-stu${NN}-key --private-key ~/c11-stu${NN}-key.pem
chmod 600 ~/c11-stu${NN}-key.pem

預期輸出:

預期輸出
+-------------+-------------------------------------------------+
| Field       | Value                                           |
+-------------+-------------------------------------------------+
| created_at  | None                                            |
| fingerprint | b5:bc:2e:d4:f3:e4:cb:49:08:b9:f7:20:c5:53:d1:b6 |
| id          | c11-stu01-key                                   |
| is_deleted  | None                                            |
| name        | c11-stu01-key                                   |
| type        | ssh                                             |
| user_id     | 947465eda0184458963bafb884996f01                |
+-------------+-------------------------------------------------+

C2. 建立開機用的 System Volume

指令
# [jmp001]
openstack volume create --image cirros --size 1 --bootable c11-stu${NN}-vol
openstack volume show c11-stu${NN}-vol -c status -c bootable -c size

預期輸出(第二個指令;status 要等到 available,約 10–20 秒,沒到就再執行一次):

預期輸出
+----------+-----------+
| Field    | Value     |
+----------+-----------+
| bootable | true      |
| size     | 1         |
| status   | available |
+----------+-----------+

C3. 建立 VM

指令
# [jmp001]
openstack server create --flavor m1.tiny --volume c11-stu${NN}-vol --network demo-net \
  --key-name c11-stu${NN}-key --wait c11-stu${NN}-vm -c status -c OS-EXT-SRV-ATTR:host -c addresses

預期輸出(約 20–30 秒):

預期輸出
+----------------------+---------------------+
| Field                | Value               |
+----------------------+---------------------+
| OS-EXT-SRV-ATTR:host | scp002-6164930-iaas |
| addresses            | demo-net=10.0.0.149 |
| status               | ACTIVE              |
+----------------------+---------------------+

C4. VM 在哪裡?經過哪些事件?

指令
# [jmp001]
openstack server show c11-stu${NN}-vm -c status -c OS-EXT-SRV-ATTR:host -c image -c volumes_attached
openstack server event list c11-stu${NN}-vm

預期輸出:

預期輸出
+----------------------+--------------------------------------------------------------------------+
| Field                | Value                                                                    |
+----------------------+--------------------------------------------------------------------------+
| OS-EXT-SRV-ATTR:host | scp002-6164930-iaas                                                      |
| image                | N/A (booted from volume)                                                 |
| status               | ACTIVE                                                                   |
| volumes_attached     | delete_on_termination='False', id='4a028183-2ed6-482e-9d1b-0b090c7ac5d1' |
+----------------------+--------------------------------------------------------------------------+
+------------------------------------------+--------------------------------------+--------+----------------------------+
| Request ID                               | Server ID                            | Action | Start Time                 |
+------------------------------------------+--------------------------------------+--------+----------------------------+
| req-df85da3e-7fe2-4f41-ada5-0716bd66dbb0 | 52c30507-d059-4c0a-a595-ebaba37b748d | create | 2026-10-31T15:51:54.000000 |
+------------------------------------------+--------------------------------------+--------+----------------------------+

把 Request ID 存成變數:

指令
# [jmp001]
REQ=$(openstack server event list c11-stu${NN}-vm -f value -c "Request ID")
echo $REQ

C5. 用 Request ID 追蹤日誌

第一站:nova-api。HAProxy 會把請求分給三台控制節點之一,所以三台都查:

指令
# [jmp001]
for i in 4 14 15; do ssh 10.127.0.$i "hostname; sudo grep -m1 $REQ /var/log/kolla/nova/nova-api.log"; done

預期輸出(只有一台會印出內容;節錄):

預期輸出
ctl001-6161060-iaas
2026-10-31 23:51:55.997 25 INFO nova.api.openstack.requestlog [None req-df85da3e-... 947465ed... b090dcce... - - default default] 10.127.0.3 "POST /v2.1/servers" status: 202 len: 376 microversion: 2.96 time: 1.897514
ctl002-6161063-iaas
ctl003-6161069-iaas

第二站:nova-compute。到 C4 看到的主機上查。主機名稱對應的 IP:scp001 = 10.127.0.7、scp002 = 10.127.0.18

指令
# [jmp001]
ssh 10.127.0.18 "sudo grep $REQ /var/log/kolla/nova/nova-compute.log"

預期輸出(節錄,省略每行開頭的 request 資訊):

預期輸出
23:51:58.360 INFO nova.compute.claims [instance: 52c30507-...] Claim successful on node scp002-6164930-iaas
23:52:02.757 INFO nova.virt.block_device [instance: 52c30507-...] Booting with volume 4a028183-... at /dev/vda
23:52:06.706 INFO os_brick.initiator.connectors.iscsi Trying to connect to iSCSI portal 10.127.0.7:3260
23:52:09.993 INFO os_vif Successfully plugged vif VIFOpenVSwitch(... bridge_name='br-int' ...)
23:52:13.192 INFO nova.compute.manager [instance: 52c30507-...] Took 7.12 seconds to spawn the instance on the hypervisor.
23:52:13.720 INFO nova.compute.manager [instance: 52c30507-...] Took 15.39 seconds to build instance.
日誌對應 p.26 的步驟
Claim successful9:compute 在本機確認資源(Placement 的 allocation 在第 6 步已由 scheduler 建立)
Booting with volume / iSCSI portal 10.127.0.7:32609:向 Cinder 取得 volume,透過 iSCSI 從 scp001 掛載
plugged vif ... br-int9:把 Neutron 的 port 接上 OVS
Took ... to spawn10:libvirt 啟動 VM

nova-scheduler 在預設日誌等級(debug = False)下,排程成功時不會寫入這個 Request ID;排程失敗(NoValidHost)時才會留下訊息。


Part D:對帳:這台 VM 用了哪裡的資源(p.16、24、27)

D1. VM 的 allocation

在 Placement 中,allocation 的 consumer 就是 server UUID(p.15)。

指令
# [jmp001]
SID=$(openstack server show c11-stu${NN}-vm -f value -c id)
openstack resource provider allocation show $SID -c resource_provider -c resources

預期輸出:

預期輸出
+--------------------------------------+-------------------------------+
| resource_provider                    | resources                     |
+--------------------------------------+-------------------------------+
| fbc35ee8-6174-5e61-b091-5037b468796b | {'MEMORY_MB': 512, 'VCPU': 1} |
+--------------------------------------+-------------------------------+

D2. 主機用量的變化

指令
# [jmp001]
openstack resource provider usage show $RP2

預期輸出:

預期輸出
+----------------+-------+
| resource_class | usage |
+----------------+-------+
| VCPU           |     3 |
| MEMORY_MB      |  2048 |
| DISK_GB        |     6 |
+----------------+-------+

和 A3 記下的數字比較:VCPU +1、MEMORY_MB +512、DISK_GB 不變。全班同時操作時,你看到的增加量會包含其他同學的 VM。

為什麼 VM 都落在 scp002? 兩台在 Part B 都是候選,接著由 weigher 排序(p.27)。RAM weigher 預設 multiplier 為 1.0,偏好剩餘 RAM 多的主機(spread);scp002 剩約 62 GB、scp001 只剩約 15 GB,所以 scp002 勝出。全班 20 台 m1.tiny 大多會落在 scp002。


Part E:生命週期操作與 allocation(p.6、7)

E1. 停止:allocation 不變

指令
# [jmp001]
openstack server stop c11-stu${NN}-vm
openstack server show c11-stu${NN}-vm -c status -c OS-EXT-SRV-ATTR:host

預期輸出(status 要等到 SHUTOFF,約 10–20 秒,沒到就再執行一次 server show):

預期輸出
+----------------------+---------------------+
| Field                | Value               |
+----------------------+---------------------+
| OS-EXT-SRV-ATTR:host | scp002-6164930-iaas |
| status               | SHUTOFF             |
+----------------------+---------------------+
指令
# [jmp001]
openstack resource provider allocation show $SID -c resource_provider -c resources

預期輸出:

預期輸出
+--------------------------------------+-------------------------------+
| resource_provider                    | resources                     |
+--------------------------------------+-------------------------------+
| fbc35ee8-6174-5e61-b091-5037b468796b | {'MEMORY_MB': 512, 'VCPU': 1} |
+--------------------------------------+-------------------------------+

VM 關機了,但仍綁在 scp002,資源繼續保留,隨時可以 server start(p.7 的 STOPPED)。

⚠️ 一定要等到 SHUTOFF 再做 E2。停止還在進行中就 shelve,會得到 409 ... while it is in task_state powering-off。

E2. 擱置(shelve):allocation 被移除

指令
# [jmp001]
openstack server shelve --wait c11-stu${NN}-vm
openstack server show c11-stu${NN}-vm -c status -c OS-EXT-SRV-ATTR:host
openstack resource provider allocation show $SID

預期輸出:

預期輸出
+----------------------+-------------------+
| Field                | Value             |
+----------------------+-------------------+
| OS-EXT-SRV-ATTR:host | None              |
| status               | SHELVED_OFFLOADED |
+----------------------+-------------------+

E3. 取消擱置(unshelve):重新排程

指令
# [jmp001]
openstack server unshelve --wait c11-stu${NN}-vm
openstack server show c11-stu${NN}-vm -c status -c OS-EXT-SRV-ATTR:host
openstack resource provider allocation show $SID -c resource_provider -c resources
openstack server event list c11-stu${NN}-vm

預期輸出:

預期輸出
+----------------------+---------------------+
| Field                | Value               |
+----------------------+---------------------+
| OS-EXT-SRV-ATTR:host | scp002-6164930-iaas |
| status               | ACTIVE              |
+----------------------+---------------------+
+--------------------------------------+-------------------------------+
| resource_provider                    | resources                     |
+--------------------------------------+-------------------------------+
| fbc35ee8-6174-5e61-b091-5037b468796b | {'MEMORY_MB': 512, 'VCPU': 1} |
+--------------------------------------+-------------------------------+
+------------------------------------------+--------------------------------------+----------+----------------------------+
| Request ID                               | Server ID                            | Action   | Start Time                 |
+------------------------------------------+--------------------------------------+----------+----------------------------+
| req-aad87c57-fc4e-4151-a8fe-da84f7e68d98 | 52c30507-d059-4c0a-a595-ebaba37b748d | unshelve | 2026-10-31T15:53:48.000000 |
| req-d2014374-faea-4c1a-913f-f9a52cd79f68 | 52c30507-d059-4c0a-a595-ebaba37b748d | shelve   | 2026-10-31T15:53:31.000000 |
| req-4bc2f33c-29a1-48d1-a290-14ee05604788 | 52c30507-d059-4c0a-a595-ebaba37b748d | stop     | 2026-10-31T15:52:33.000000 |
| req-df85da3e-7fe2-4f41-ada5-0716bd66dbb0 | 52c30507-d059-4c0a-a595-ebaba37b748d | create   | 2026-10-31T15:51:54.000000 |
+------------------------------------------+--------------------------------------+----------+----------------------------+
操作vm_statehostPlacement allocation
createACTIVEscp002建立
stopSTOPPED(STATUS 顯示 SHUTOFF)scp002不變
shelveSHELVED_OFFLOADEDNone移除
unshelveACTIVE重新排程重新建立
deleteDELETED—移除(Part F)

Part F:清除

指令
# [jmp001]
openstack server delete --wait c11-stu${NN}-vm
openstack resource provider allocation show $SID
openstack volume delete c11-stu${NN}-vol
openstack keypair delete c11-stu${NN}-key
rm -f ~/c11-stu${NN}-key.pem
openstack server list --name c11-stu${NN}
openstack volume list --name c11-stu${NN}-vol

預期輸出:(全部沒有輸出)

預期輸出