Incremental tables not syncing with Elasticsearch due to index not found error

Hi team,

Could you please let me know if incremental tables (for example aws_cloudtrail_events) should work with a destination like Elasticsearch? I tried this using AWS as a source and Elasticsearch as the destination with the latest versions, but I encountered an error (see below). The backend_options table_name is “cq_state_aws”. Do you have any ideas, please?

----------error log:
Error: failed to sync v3 source aws: unexpected error from sync client receive: rpc error: code = Unknown desc = failed to sync records: failed to sync unmanaged client: failed to create state client: rpc error: code = Unknown desc = failed to sync records: failed to read: failed to read: 404 Not Found: {"error":{"root_cause":[{"type":"index_not_found_exception","reason":"no such index [cq_state_aws]","resource.type":"index_or_alias","resource.id":"cq_state_aws","index_uuid":"_na_","index":"cq_state_aws"}],"type":"index_not_found_exception","reason":"no such index [cq_state_aws]","resource.type":"index_or_alias","resource.id":"cq_state_aws","index_uuid":"_na_","index":"cq_state_aws"},"status":404}

Hi, ElasticSearch should support incremental tables. Would you mind opening an issue on GitHub with the appropriate details so we can triage and fix it?

Open an issue on GitHub