spark SaveMode数据源写入模式

spark | 2019-09-13 10:02:39

整理一下spark SaveMode数据源写入模式，包括 异常 追加 覆盖 存在忽略


Scala/JavaAny LanguageMeaning
SaveMode.ErrorIfExists(default)"error" or "errorifexists"(default)

When saving a DataFrame to a data source, if data already exists, an exception is expected to be thrown

.如果存在就抛出异常

SaveMode.Append"append"

When saving a DataFrame to a data source, if data/table already exists, contents of the DataFrame are expected to be appended to existing data

.直接追加

SaveMode.Overwrite"overwrite"

Overwrite mode means that when saving a DataFrame to a data source, if data/table already exists, existing data is expected to be overwritten by the contents of the DataFrame

.覆盖数据

SaveMode.Ignore"ignore"

Ignore mode means that when saving a DataFrame to a data source, if data already exists, the save operation is expected to not save the contents of the DataFrame and to not change the existing data. This is similar to a CREATE TABLE IF NOT EXISTS in SQL

.数据已经存在就忽略，不做任何操作



登录后即可回复 登录 | 注册
    
相关文章
spark操作hive orc transactional事务表异常解决spark hive插入数据异常spark currently does not populate bucketed outputspark savemode数据源写入模式spark dataset写入hive表spark异常 could not locate executable null bin winutils.exe in the hadoop binariesspark 基础概念和运行原理spark sql 实现 group concat分组拼接列值解决spark shell 写入hbase 异常job in state define instead of runningspark rdd写入数据到hbase nullpointerexception异常spark从oracle导入数据到hivespark scala中 = 符号的使用场景scala match case 模式匹配php bcompiler 函数 写入 php 源码文件的字节码php ftp 函数 返回当前 ftp 被动模式是否打开php mcrypt 函数 打开算法和模式对应的模块php postgresql 函数 向大型对象写入数据php 以 cgi 模式安装时 情形三：设置 doc root 或 user dirspark broadcast广播变量详解spark写入mongo 按多维度提取分层jsonspark多线程并发执行job
关注编程学问公众号