1

How can I download a file in DBFS to my local system ? The file was created from a Databricks Notebook.

CHEEKATLAPRADEEP-MSFT
  • 11,445
  • 1
  • 14
  • 35
learner
  • 533
  • 2
  • 6
  • 18

2 Answers2

4

Method1: Using Databricks portal GUI, you can download full results (max 1 millions rows).

enter image description here

To download full results, first save the file to dbfs and then copy the file to local machine using Databricks cli as follows.

dbfs cp "dbfs:/FileStore/tables/my_my.csv" "A:\AzureAnalytics"

You can access DBFS objects using the DBFS CLI, DBFS API, Databricks file system utilities (dbutils.fs), Spark APIs, and local file APIs.

In a Spark cluster you access DBFS objects using Databricks file system utilities, Spark APIs, or local file APIs.

On a local computer you access DBFS objects using the Databricks CLI or DBFS API.

Reference: Azure Databricks – Access DBFS

Method2: Using Databricks CLI

The DBFS command-line interface (CLI) uses the DBFS API to expose an easy to use command-line interface to DBFS. Using this client, you can interact with DBFS using commands similar to those you use on a Unix command line. For example:

# List files in DBFS
dbfs ls
# Put local file ./apple.txt to dbfs:/apple.txt
dbfs cp ./apple.txt dbfs:/apple.txt
# Get dbfs:/apple.txt and save to local file ./apple.txt
dbfs cp dbfs:/apple.txt ./apple.txt
# Recursively put local dir ./banana to dbfs:/banana
dbfs cp -r ./banana dbfs:/banana

Reference: Installing and configuring Azure Databricks CLI

Method3: Using third-party tool named DBFS Explorer

DBFS Explorer was created as a quick way to upload and download files to the Databricks filesystem (DBFS). This will work with both AWS and Azure instances of Databricks. You will need to create a bearer token in the web interface in order to connect.

enter image description here

CHEEKATLAPRADEEP-MSFT
  • 11,445
  • 1
  • 14
  • 35
1

Follow the discussion here

Or you can simply setup databricks cli and using the dbfs cli you can copy files to and from dbfs easily.

For more information about the command follow the official documentation: https://docs.databricks.com/dev-tools/cli/dbfs-cli.html

Shubham Jain
  • 4,822
  • 2
  • 12
  • 31