scala - What are Spark UDFS? -
i've been writing transformers , estimators , keep noticing udfs. i've read through source , have general sense of hoping give me high level explanation.
what have gleaned code can create udf , apply such runs on each row in dataframe particular column. guess curious why way?
all custom manipulation write on rdds , dataframes "user defined functions". register udf though you'd able use in select statements used in spark-sql (sqlcontext.sql("select myudf(fieldname) myregistreddf")...
Comments
Post a Comment