Server Fault Stack Exchange
Server Fault Stack Exchange

Questions tagged [ceph]

Ceph is a free software storage platform designed to present object, block, and file storage from a single distributed computer cluster.

Ceph is a free software storage platform designed to present object, block, and file storage from a single distributed computer cluster. Ceph's main goals are to be completely distributed without a single point of failure, scalable to the exabyte level, and freely-available.

202 questions
4
votes
2 answers

Ceph does not detect disks

I am using cephadm. I had a test cluster up and running. I re-installed the OS and during installation nuked the drives by deleting the LVM volume groups and partitions. Now lsblk shows the devices to be present but ceph orch device ls returns…
Tintin
  • 193
4
votes
0 answers

How to create Ceph OSDs on Partitions using cephadm, on ceph-pacific?

In the pacific version of Ceph, OSD creation is not allowing by cephadm on partitions.(We have NvMes with partitions). The command used is: ceph orch daemon add osd : The above command only accepts, disks or lvm, and throws error…
Arun Vinod
  • 51
3
votes
1 answer

Slow fsync() with ceph (cephfs)

I have built an experimental ceph cluster - 12 nodes, 50 osds, 3 mons, 3 mds, for which I'm trying to run a samba gateway. It seems that when writing lots of small files, the fsync() system calls from samba will routinely block, presumably at the…
kdm
  • 31
3
votes
1 answer

ceph osd down and rgw Initialization timeout, failed to initialize after reboot

Centos7.2, Ceph with 3 OSD, 1 MON running on a same node. radosgw and all the daemons are running on the same node, and everything was working fine. After reboot the server, all osd could not communicate (looks like) and the radosgw does not work…
Tiina
  • 175
3
votes
1 answer

Ceph e5 handle_auth_request failed to assign global_id after a host outage

I have a small 3-host Ceph cluster with Ubuntu 20.04.1 and Ceph 15.2.5 using docker containers and deployed with cephadm. Yesterday one of the hosts (s65-ceph) had a power outage. The other two hosts continued working for a while but then s63-ceph…
Paolo Celati
  • 71
3
votes
0 answers

Ceph Rados GW Proxy

I've been given an access key to a Ceph cluster that runs radoswgw to provide S3. The key allows bucket creation, object reading etc. I don't admin the Ceph or radosgw. I have a group of users that want to use the Ceph object store, but need data…
Chris
  • 131
2
votes
0 answers

ceph shows wrong USED space in a single replicated pool

We are using ceph version 14.2.0. We have 4 hosts with 24 BlueStore OSDs, each is 1.8TB (2TB spinning disk). We have only a single pool with size 2 and I am absolutely sure that we are using more space than what ceph df shows: [root@blackmirror ~]#…
Jacket
  • 131
2
votes
1 answer

On what network do the clients connect to the CEPH cluster (public/private)?

It's recommended to have a public network and cluster network when setting up CEPH. For what I understand this cluster network is what the nodes use to replicate data accross so that would preferably be a 10 gigabit network. However I read that only…
Maarten Ureel
  • 239
2
votes
0 answers

What is the correct usage of ceph-objectsore-tool?

I am trying to export data for pg, but am getting errors from ceph-objectstore-tool. I see some examples online of usage which seem to match this, and reading through the usage statement that the tool generates does not at all clarify what is wrong…
blitzen9872
  • 121
2
votes
1 answer

How do I mount one of multiple filesystems in a ceph cluster?

Ceph now includes (experimental) support for multiple filesystems within a single storage cluster; but the mount options don't seem to allow specifying which filesystem to mount. I have configured two testing filesystems, each with their own mds and…
rvalue
  • 121
2
votes
1 answer

Using available space in a Ceph pool

Here's what my Ceph situation looks like (from ceph df): GLOBAL: SIZE AVAIL RAW USED %RAW USED 596G 593G 3633M 0.59 POOLS: NAME ID USED %USED MAX AVAIL OBJECTS …
Turtle V. Rabbit
  • 183
2
votes
0 answers

Conditions for "mds cluster is degraded"

I see into Ceph and after running command $ ceph status I meet text "mds cluster is degraded". I try to seek it at official Ceph docs, but nothing find. If I understand correctly, MDS needed for mounting and presenting ceph's data as regular FS. Can…
Lazy Starwing
  • 21
2
votes
1 answer

Where does CEPH store device metrics for each device? [ceph reef]

I just deployed my first ceph cluster and have been trying multiple things so far. Though, I couldn't understand how ceph store metric for the devices. I can easily access those through: # ceph device get-health-metrics --help Even though it…
Robson Jr
  • 51
2
votes
1 answer

bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (1) Operation not permitted

When I have upgrade the from Octopus 15.2.10 to Pacific 16.2.0, the mon nodes starts successfully using the manual upgrade process (by installing the packages with no orch) however when I have upgrade the OSDs, the ceph-osd service does not…
user393397
  • 37
1
vote
0 answers

ceph-deploy osd create --data /dev/vdb fails

From my ceph (mimic release) administrative node I ran ceph-deploy osd create --data /dev/vdb ceph0 against a bare metal ceph node and it worked without error. Now I run the same command against a virtual ceph node (with an unpartitioned qcow2…
mr.zog
  • 941
  • 3
  • 20
  • 40
1
2 3