-1

My Data looks like this

H1234|1234|1999-12-03.3.22.34.132456
G1345|2345|1998-11-03-12.22.45.23456

I stored this data on a List[String], while converting it to a data frame by doing:

val dataframe = list.map(r => r.split("\\|")).map(r => (r(0),r(1),r(2)).toDF("ID","Number","Timestamp ")

but when I am using dataframe.show, the Timestamp Column, I am getting it as below:

(FYI every value is a String)

Timestamp
1999-12-03.3.22....
1998-11-03-12.22...

Could you please tell me how to solve this.

Tomer Shetah
  • 8,103
  • 7
  • 23
  • 34
Sidarth Bazaz
  • 13
  • 4
  • I guess this is what you are looking for: https://stackoverflow.com/questions/41867147/convert-list-into-dataframe-spark-scala – Michael Heil Dec 16 '20 at 16:04

1 Answers1

1

Your data is actually intact, but show is truncating the output. Do dataframe.show(false) to avoid truncation of output.

mck
  • 37,331
  • 13
  • 29
  • 45
  • @mike I saw your link, but given that the OP got a timestamp column with two rows, I suspect the OP managed to build a proper dataframe... anyway let's wait for the OP's response – mck Dec 16 '20 at 16:06
  • I just wrote a little bit of code where i am applying transformation.... If you want I can write the whole code. I just wanted to give you guys a basic idea of what my data looks like and what problem I faced @mck – Sidarth Bazaz Dec 16 '20 at 16:48