Searching...
miércoles, 12 de julio de 2023

Transformar dataframe spark

 def transformArrayStruct(df: DataFrame): DataFrame = {

df.withColumn("array_of_struct", expr("transform(array_of_struct, x -> struct(x.a, x.c))")) }



import org.apache.spark.sql.functions.{col, isArray, isStruct} import org.apache.spark.sql.{DataFrame, SparkSession} def isColumnArrayStruct(df: DataFrame, columnName: String): Boolean = { val arrayCheck = isArray(col(columnName)) val structCheck = isStruct(element_at(col(columnName), 1)) df.select(arrayCheck && structCheck).as[Boolean].head() }

0 comentarios:

Publicar un comentario

Gracias por participar en esta página.

:) :)) ;(( :-) =)) ;( ;-( :d :-d @-) :p :o :>) (o) [-( :-? (p) :-s (m) 8-) :-t :-b b-( :-# =p~ $-) (b) (f) x-) (k) (h) (c) cheer
Click to see the code!
To insert emoticon you must added at least one space before the code.

 
Back to top!