spark scala map转 rdd 转 dataframe

2020-11-06 17:06:52 | 编辑

spark scala map转 rdd 然后转 dataframe的方法,主要用到seq的toDF方法

    var subjectScoreLevelDetail:Seq[Map[String,String]]=Seq(Map("subjectId"->"001","excellent"->"90","good"->"80","pass"->"60"),Map("subjectId"->"002","excellent"->"90","good"->"80","pass"->"60"))

    var s=subjectScoreLevelDetail.map(kv=>{(kv.get("subjectId"),kv.get("excellent").get.toDouble,kv.get("good").get.toDouble,kv.get("pass").get.toDouble)})

    var dataframe=s.toDF("subjectId","excellent","good","pass")
    dataframe.show

 

 

 

登录后即可回复 登录 | 注册
    
关注编程学问公众号