How to download docker image using HTTP API using docker hub credentials

Question

So I have a private repository at docker hub and I am trying to download image (blobs) manually using HTTP API.

Now there are some issues

I have tried https://hub.docker.com/support/doc/how-do-i-authenticate-with-the-v2-api and this script works and I can see my tags.

But there is no API in docker HUB api to get list of blobs from a tag and then download it.

There is a docker registry api, but there my username password does not work. What to do?

[How do I download Docker images without using the pull command?](https://stackoverflow.com/questions/37905763/how-do-i-download-docker-images-without-using-the-pull-command) could help, if you look into implementations of the tools. — Lei Yang, Mar 09 '22 at 12:46
do you think the docker registry really support http download? if api doesn't support, then you can't. i suggest you start with a self host private registry, using http. then you can use some tools such as wireshark to capture the traffic. — Lei Yang, Mar 09 '22 at 23:34

score 1 · Answer 1 · answered Mar 17 '22 at 14:30

For an image, you first pull the manifest and parse that for the list of blobs that you need to pull. All of these API's need the same authorization headers used to list the tags. I'm going to be using regctl from my regclient project to query a local registry, but you could also use curl against Hub which I show below.

$ regctl tag ls localhost:5000/library/alpine
3
3-bkup-20210904
3.10
3.11
3.12
3.13
3.14
3.2
3.3
3.4
3.5
3.6
3.7
3.8
3.9
latest

$ regctl manifest get localhost:5000/library/alpine:latest --format body | jq .
{
  "manifests": [
    {
      "digest": "sha256:14b55f5bb845c7b810283290ce057f175de87838be56f49060e941580032c60c",
      "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
      "platform": {
        "architecture": "amd64",
        "os": "linux"
      },
      "size": 528
    },
    {
      "digest": "sha256:40f396779ba29da16f29f780963bd4ad5b7719e3eb5dec04516d583713256aa8",
      "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
      "platform": {
        "architecture": "arm",
        "os": "linux",
        "variant": "v6"
      },
      "size": 528
    },
    {
      "digest": "sha256:392d9d85dff31e34d756be33579f05ef493cb1b0edccc36a11b3295365553bfd",
      "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
      "platform": {
        "architecture": "arm",
        "os": "linux",
        "variant": "v7"
      },
      "size": 528
    },
    {
      "digest": "sha256:4fb53f12d2ec18199f16d7c305a12c54cda68cc622484bfc3b7346a44d5024ac",
      "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
      "platform": {
        "architecture": "arm64",
        "os": "linux",
        "variant": "v8"
      },
      "size": 528
    },
    {
      "digest": "sha256:e8d9cf28250078f08e890a3466efbefda68a8feac03cc4076d3ada3397370d6e",
      "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
      "platform": {
        "architecture": "386",
        "os": "linux"
      },
      "size": 528
    },
    {
      "digest": "sha256:d860569a59af627dafee0b0f2b8069e31b07fbdaebe552904dbaec28047ccf64",
      "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
      "platform": {
        "architecture": "ppc64le",
        "os": "linux"
      },
      "size": 528
    },
    {
      "digest": "sha256:6640b198347e5bf1e9a9dc5fc864e927154275dc31f3d26193b74350a5c94c9f",
      "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
      "platform": {
        "architecture": "s390x",
        "os": "linux"
      },
      "size": 528
    }
  ],
  "mediaType": "application/vnd.docker.distribution.manifest.list.v2+json",
  "schemaVersion": 2
}

$ regctl manifest get localhost:5000/library/alpine@sha256:14b55f5bb845c7b810283290ce057f175de87838be56f49060e941580032c60c --format body | jq .
{
  "schemaVersion": 2,
  "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
  "config": {
    "mediaType": "application/vnd.docker.container.image.v1+json",
    "size": 1472,
    "digest": "sha256:e9adb5357e84d853cc3eb08cd4d3f9bd6cebdb8a67f0415cc884be7b0202416d"
  },
  "layers": [
    {
      "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
      "size": 2812636,
      "digest": "sha256:3d243047344378e9b7136d552d48feb7ea8b6fe14ce0990e0cc011d5e369626a"
    }
  ]
}

$ regctl blob get localhost:5000/library/alpine sha256:e9adb5357e84d853cc3eb08cd4d3f9bd6cebdb8a67f0415cc884be7b0202416d | jq .
{
  "architecture": "amd64",
  "config": {
    "Hostname": "",
    "Domainname": "",
    "User": "",
    "AttachStdin": false,
    "AttachStdout": false,
    "AttachStderr": false,
    "Tty": false,
    "OpenStdin": false,
    "StdinOnce": false,
    "Env": [
      "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
    ],
    "Cmd": [
      "/bin/sh"
    ],
    "Image": "sha256:e211ac20c5c7aaa4ed30d5553654d4679082ec48efcb4d164bac6d50d62653fd",
    "Volumes": null,
    "WorkingDir": "",
    "Entrypoint": null,
    "OnBuild": null,
    "Labels": null
  },
  "container": "b6ba94212561a8075e1d324fb050db160e25035ffcfbbe5b410e411e2b7000e2",
  "container_config": {
    "Hostname": "b6ba94212561",
    "Domainname": "",
    "User": "",
    "AttachStdin": false,
    "AttachStdout": false,
    "AttachStderr": false,
    "Tty": false,
    "OpenStdin": false,
    "StdinOnce": false,
    "Env": [
      "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
    ],
    "Cmd": [
      "/bin/sh",
      "-c",
      "#(nop) ",
      "CMD [\"/bin/sh\"]"
    ],
    "Image": "sha256:e211ac20c5c7aaa4ed30d5553654d4679082ec48efcb4d164bac6d50d62653fd",
    "Volumes": null,
    "WorkingDir": "",
    "Entrypoint": null,
    "OnBuild": null,
    "Labels": {}
  },
  "created": "2022-03-17T04:01:59.188838147Z",
  "docker_version": "20.10.12",
  "history": [
    {
      "created": "2022-03-17T04:01:58.883733237Z",
      "created_by": "/bin/sh -c #(nop) ADD file:cf4b631a115c2bbfbd81cad2d3041bceb64a8136aac92ba8a63b6c51d60af764 in / "
    },
    {
      "created": "2022-03-17T04:01:59.188838147Z",
      "created_by": "/bin/sh -c #(nop)  CMD [\"/bin/sh\"]",
      "empty_layer": true
    }
  ],
  "os": "linux",
  "rootfs": {
    "type": "layers",
    "diff_ids": [
      "sha256:5e03d8cae8773cb694fff1d55da34a40d23c2349087ed15ce68476395d33753c"
    ]
  }
}

$ regctl blob get localhost:5000/library/alpine sha256:3d243047344378e9b7136d552d48feb7ea8b6fe14ce0990e0cc011d5e369626a | tar -tvzf - | head
drwxr-xr-x 0/0               0 2022-03-16 16:15 bin/
lrwxrwxrwx 0/0               0 2022-03-16 16:15 bin/arch -> /bin/busybox
lrwxrwxrwx 0/0               0 2022-03-16 16:15 bin/ash -> /bin/busybox
lrwxrwxrwx 0/0               0 2022-03-16 16:15 bin/base64 -> /bin/busybox
lrwxrwxrwx 0/0               0 2022-03-16 16:15 bin/bbconfig -> /bin/busybox
-rwxr-xr-x 0/0          824984 2022-02-02 13:21 bin/busybox
lrwxrwxrwx 0/0               0 2022-03-16 16:15 bin/cat -> /bin/busybox
lrwxrwxrwx 0/0               0 2022-03-16 16:15 bin/chgrp -> /bin/busybox
lrwxrwxrwx 0/0               0 2022-03-16 16:15 bin/chmod -> /bin/busybox
lrwxrwxrwx 0/0               0 2022-03-16 16:15 bin/chown -> /bin/busybox
...

A few examples if you try doing this with curl:

Get a token (specific to Docker Hub, each registry may have different auth methods and servers):

token=$(curl -s "https://auth.docker.io/token?service=registry.docker.io&scope=repository:${repo}:pull" \
        | jq -r '.token')

Tags:

curl -H "Authorization: Bearer $token" \
     -s "https://registry-1.docker.io/v2/${repo}/tags/list" | jq .

Manifests:

api="application/vnd.docker.distribution.manifest.v2+json"
apil="application/vnd.docker.distribution.manifest.list.v2+json"
curl -H "Accept: ${api}" -H "Accept: ${apil}" \
     -H "Authorization: Bearer $token" \
     -s "https://registry-1.docker.io/v2/${repo}/manifests/${sha:-$tag}" | jq .

And blobs:

curl -H "Authorization: Bearer $token" \
     -s -L -o - "https://registry-1.docker.io/v2/${repo}/blobs/${digest}"

If you want to export everything to be able to later import, regctl image export will convert the image to a tar of manifests and blobs. This is output in the OCI Layout format, and if you only pull a single image manifest (and not a multi-platform manifest) it will also include the needed files for docker load to import the tar.

How to download docker image using HTTP API using docker hub credentials

1 Answers1