Searching...
miércoles, 12 de julio de 2023

Transformar dataframe spark

 def transformArrayStruct(df: DataFrame): DataFrame = {

df.withColumn("array_of_struct", expr("transform(array_of_struct, x -> struct(x.a, x.c))")) }



import org.apache.spark.sql.functions.{col, isArray, isStruct} import org.apache.spark.sql.{DataFrame, SparkSession} def isColumnArrayStruct(df: DataFrame, columnName: String): Boolean = { val arrayCheck = isArray(col(columnName)) val structCheck = isStruct(element_at(col(columnName), 1)) df.select(arrayCheck && structCheck).as[Boolean].head() }

0 comentarios:

Publicar un comentario

Gracias por participar en esta página.

 
Back to top!