How to check if a partition exists in Hive?

Question

I have a Hive table, which is partitioned by column dt. I need to add a partition if it does not exists, for exmaple, dt='20181219'.

Now I'm using HiveMetaStoreClient#getPartition(dbName, tableName, 20181219). If the partition does not exists, then catch NoSuchObjectException and add it.

Is there any elegant way to achieve this in Java?

Try this: https://stackoverflow.com/a/50174062/2700344 – leftjoin Dec 19 '18 at 14:53 — leftjoin, Dec 19 '18 at 14:53

Stephen C · Accepted Answer · 2018-12-29T04:41:39.460

Use add_partition(Partition, ifNotExists, needResults) (javadoc) ... which (if the 2nd argument is true) will only create an partition if it doesn't already exist.

Alternatively, just use add_partition(Partition) to add the partition without a test, and catch the AlreadyExistsException if it occurs.

Any scheme that involves testing that an action is possible and then doing that action has a potential race condition. In between the "test" and the "do", some other agent (thread, client, whatever) could do an action (the same one or a different one) that will cause your attempt to fail.

So your current approach is not only ugly. It also has a potential race condition if partitions could be created by multiple agents.

How to check if a partition exists in Hive?

1 Answers1