Skip to content

Conversation

@subkanthi
Copy link
Contributor

closes: #616

@github-actions github-actions bot added the INFRA label Jan 15, 2026
@subkanthi subkanthi marked this pull request as draft January 20, 2026 18:19
@subkanthi
Copy link
Contributor Author

The current implementation of commitTable is not atomic, using Hive Locking API similar to iceberg-python

        # https://github.com/apache/hive/blob/master/standalone-metastore/metastore-common/src/main/thrift/hive_metastore.thrift#L1232
        with self._client as open_client:
            lock: LockResponse = open_client.lock(self._create_lock_request(database_name, table_name))

            try:
                if lock.state != LockState.ACQUIRED:
                    if lock.state == LockState.WAITING:
                        self._wait_for_lock(database_name, table_name, lock.lockid, open_client)
                    else:
                        raise CommitFailedException(f"Failed to acquire lock for {table_identifier}, state: {lock.state}")

@subkanthi subkanthi marked this pull request as ready for review January 23, 2026 00:03
Copy link
Member

@zeroshade zeroshade left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for the long delay here, it's been a very busy couple weeks. Here's my first pass, more comments to come later tonight/tomorrow

)

type HiveClient interface {
Close()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not using an io.Closer where the Close method returns error?

"github.com/beltran/gohive/hive_metastore"
)

type HiveClient interface {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is there any particular reason to export this? Is the intent that users should be able to provide their own HiveClient if they want?

client *gohive.HiveMetastoreClient
}

func NewHiveClient(uri string, opts *HiveOptions) (HiveClient, error) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we expect users to actually be using this directly as opposed to just using NewCatalog?

// Determine authentication mode
auth := "NOSASL"
if opts != nil && opts.KerberosAuth {
auth = "KERBEROS"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I'm reading gohive correctly, this requires the kerberos tag to build correctly, right? Can we instead use one of the pure go implementations that don't rely on cgo? Like gokrb etc.


func (c *thriftClient) Close() {
if c.client != nil {
c.client.Close()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't we set c.client = nil after we close?

}

type thriftClient struct {
client *gohive.HiveMetastoreClient
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Most of the methods here just forward to client, should we just embed the client rather than have it be a member?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

When is it expected that Hive catalog support will be available?

2 participants