3

I have a csv file that i want to render in a fastAPI app. I only managed to render te csv in json format like following:

def transform_question_format(csv_file_name):

    json_file_name = f"{csv_file_name[:-4]}.json"

    # transforms the csv file into json file
    pd.read_csv(csv_file_name ,sep=",").to_json(json_file_name)

    with open(json_file_name, "r") as f:
        json_data = json.load(f)

    return json_data

@app.get("/questions")
def load_questions():

    question_json = transform_question_format(question_csv_filename)

    return question_json

When i tried directly pd.read_csv(csv_file_name ,sep=",").to_json(json_file_name) in return it does work as it returns a string. How should I proceed ? I believe this is not the good way to do it.

pac
  • 385
  • 1
  • 11
  • 1
    When you say `render` - what do you mean? In general, FastAPI returns data as JSON. If you want to have a different response format, you can use one of the built-in custom response formats, or create your own: https://fastapi.tiangolo.com/advanced/custom-response/ – MatsLindh Feb 21 '22 at 09:19
  • 1
    Maybe check this https://stackoverflow.com/questions/32911336/what-is-the-difference-between-json-dumps-and-json-load, but so far it seems good – Alsushi Feb 21 '22 at 09:40
  • I am ok with JSON output but problem is that i need this intermediate step of creating an output JSON file and then load it. Obviously i cannot import csv, transform and load in one step. thanks for the links. It clarifies a bit the process. – pac Feb 21 '22 at 10:13
  • 1
    If you don't give a filename to `to_json` a JSON string is returned directly. You can then pair this with `return Response(content=json_str, media_type="application/json")` to return the string directly from FastAPI with a JSON header. Would that work? (you can also give a File-like object and get output written to that, so something like `StringIO` should work as well) – MatsLindh Feb 21 '22 at 11:05

1 Answers1

1

The below shows four different ways to return the data from a csv file.

Option 1 is to convert the file data into JSON and then parse it into a dict. You can optionally change the orientation of the data using the orient parameter in the to_json() method.

  • Update 1: Using to_dict() method might be a better option, as there is no need for parsing the JSON string.
  • Update 2: When using to_dict() method and returning the dict, FastAPI, behind the scenes, automatically converts that return value into JSON, using the jsonable_encoder. Thus, to avoid that extra work, you could still use to_json() method, but instead of parsing the JSON string, put it in a Response and return it directly, as shown in the example below.

Option 2 is to return the data in string format, using to_string() method.

Option 3 is to return the data as an HTML table, using to_html() method.

Option 4 is to return the file as is using FastAPI's FileResponse.

from fastapi import FastAPI, Response
from fastapi.responses import FileResponse
from fastapi.responses import HTMLResponse
import pandas as pd
import json

df = pd.read_csv("file.csv")
app = FastAPI()

def parse_csv(df):
    res = df.to_json(orient="records")
    parsed = json.loads(res)
    return parsed
    
@app.get("/questions")
def load_questions():
    return Response(df.to_json(orient="records"), media_type="application/json")  # Option 1 (Updated 2): Return as JSON directly
    #return df.to_dict(orient="records")  # Option 1 (Updated 1): Return as dict (encoded to JSON behind the scenes)
    #return parse_csv(df)  # Option 1: Parse the JSON string and return as dict (encoded to JSON behind the scenes)
    #return df.to_string()  # Option 2: Return as string
    #return HTMLResponse(content=df.to_html(), status_code=200)  # Option 3: Return as HTML Table
    #return FileResponse(path="file.csv", filename="file.csv")   # Option 4: Return as File
Chris
  • 4,940
  • 2
  • 7
  • 28
  • You can avoid json dump/load sequence by just calling `DataFrame.to_dict()` instead. – Tzane Feb 21 '22 at 11:27
  • @Chris by the way i saw you updated with `async`, and of course i have seen this into the documentation. In such a case would it be to allow several users to query the API at the same time ? – pac Feb 21 '22 at 13:13
  • 1
    @pac `async` is not really needed in this case. Removed it. You could read more about `async` [here](https://fastapi.tiangolo.com/async/). – Chris Feb 21 '22 at 13:17
  • 1
    @pac You could also have a look at [this](https://stackoverflow.com/a/71188190/17865804) answer, if it helps clarify things about `async` for you. – Chris Feb 21 '22 at 13:24