7

I recently provisioned 10 m5d.large instances (with Terrafor. The advertised space is "1 x 75 NVMe SSD", however when I ssh on to the instance I see.

[root@web0 ~]# df -h
Filesystem      Size  Used Avail Use% Mounted on
devtmpfs        3.8G     0  3.8G   0% /dev
tmpfs           3.8G     0  3.8G   0% /dev/shm
tmpfs           3.8G  376K  3.8G   1% /run
tmpfs           3.8G     0  3.8G   0% /sys/fs/cgroup
/dev/nvme0n1p1   30G  2.5G   27G   9% /
tmpfs           777M     0  777M   0% /run/user/1000
tmpfs           777M     0  777M   0% /run/user/0

I also see in the AWS console 10 Elastic Block volumes of 30 GB.

Why am I not getting the 75 GB advertised https://aws.amazon.com/ec2/instance-types/m5/?

Update

Running fdisk shows the same

[ec2-user@web0 ~]$ sudo fdisk -l /dev/nvme0n1
Disk /dev/nvme0n1: 30 GiB, 32212254720 bytes, 62914560 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 324AAE3D-30C7-46AB-9185-9756BFACB090

Device           Start      End  Sectors Size Type
/dev/nvme0n1p1    4096 62914526 62910431  30G Linux filesystem
/dev/nvme0n1p128  2048     4095     2048   1M BIOS boot


Partition table entries are not in disk order.

Update

Running lsblk does show something

[ec2-user@web0 ~]$ lsblk
NAME          MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
nvme1n1       259:0    0 69.9G  0 disk
nvme0n1       259:1    0   30G  0 disk
├─nvme0n1p1   259:2    0   30G  0 part /
└─nvme0n1p128 259:3    0    1M  0 part

Not sure what is going on here, this is a vanilla provision with Terraform, perhaps I just need to mount the 69 GB disk. Can't see where this 30 GB EBS is coming from or why its mounted on /.

Niall Gallagher
  • 83
  • Did you happen to ask explicitly for 30GB disks in your terraform configuration? – Michael Hampton Jun 17 '21 at 22:43
  • I did not specifically request it. When doing a terraform plan I see that the root_block_device is "(known after apply)". When I dump out the JSON (created from apply) I see "throughput": 0, "volume_size": 30, "volume_type": "gp2". The same block also shows "ebs_block_device": []. – Niall Gallagher Jun 17 '21 at 22:58
  • Shot in the dark here, but https://aws.amazon.com/ebs/pricing/ says 30 GB is in the "free tier", I wonder if this is just a complementary EBS volume? If it is, ill bet there are many deployments/customers using it over the attached NVMe disk. – Niall Gallagher Jun 17 '21 at 23:01

2 Answers2

16

You've got 2 disks as lsblk confirms:

  • /dev/nvme0n1 is your EBS volume with the operating system, created from the AMI. Most likely the 30GB size is the default size prescribed by the AMI you used.

    It's not free though - you pay ca $0.10/GB/month (depending on the region and storage type)

    This is automatically mounted and used by your instance.

  • /dev/nvme1n1 is your Ephemeral storage (or Instance storage). It's faster, cheaper, but will be wiped when the instance is stopped (by you or if it crashes).

    It comes unpartitioned and unformatted and unmounted - you have to do that all yourself in your startup scripts.

    Only use it for data that you don't mind losing! E.g. cache, or something that can be recreated easily.

More info about Instance storage

Hope that helps :)

MLu
  • 25,409
5

Looks like you provisioned a 30gb ebs root disk, that's the one mounted on / and you have the added ephemeral nvme1n1 showing as well.

Format nvme1n1 and mount it wherever you need it. Keep in mind the data on such a disk will not survive a cold boot of the instance.

dyasny
  • 18,917
  • Thing is I did not provision the 30 GB EBS. It just appeared. I have no problem keeping it if its free, however I find it odd that it exists at all. – Niall Gallagher Jun 17 '21 at 23:21
  • 2
    I haven't used nvme instances, but AFAIK your OS disk is always going to be EBS as it's durable. If you don't explicitly provision the EBS disk it's probably automatic. – Tim Jun 18 '21 at 00:37
  • This is what I am thinking also. – Niall Gallagher Jun 18 '21 at 00:54
  • Exactly. Depending on the AMI there's a default root disk size provisioned, and the root disk AFAIK always comes from ebs – dyasny Jun 18 '21 at 03:45
  • Correct - there is no good way to use ephemeral disk as a boot device. It is wiped on a stop-start, while "probably" surviving a plain reboot which leaves the instance inside the same AWS droplet. – Criggie Jun 19 '21 at 00:04