3

I have been using tap-mysql from singer.io

Below is my config file:

{ "host": "localhost", "port": "3306", "user": "root", "password": "password"}

This successfully returns schema on --discover

and my properties file is:

 {
  "streams": [
    {
      "key_properties": [
        "id"
      ],
      "tap_stream_id": "example_db-animals",
      "schema": {
  "selected": "true",
  "properties": {
    "likes_getting_petted": {
      "selected": "true",
      "inclusion": "available",
      "type": [
        "null",
        "boolean"
      ]
    },
    "name": {
      "selected": "true",
      "maxLength": 255,
      "inclusion": "available",
      "type": [
        "null",
        "string"
      ]
    },
    "id": {
      "selected": "true",
      "minimum": -2147483648,
      "inclusion": "automatic",
      "maximum": 2147483647,
      "type": [
        "null",
        "integer"
      ]
    }
  },
  "type": "object"
},
      "table_name": "animals",
      "metadata": [
        {
          "metadata": {
            "selected-by-default": true,
            "sql-datatype": "int(11)"
          },
          "breadcrumb": [
            "properties",
            "id"
          ]
        },
        {
          "metadata": {
            "database-name": "example_db",
            "selected-by-default": false,
            "is-view": false,
            "row-count": 3
          },
          "breadcrumb": []
        },
        {
          "metadata": {
            "selected-by-default": true,
            "sql-datatype": "varchar(255)"
          },
          "breadcrumb": [
            "properties",
            "name"
          ]
        },
        {
          "metadata": {
            "selected-by-default": true,
            "sql-datatype": "tinyint(1)"
          },
          "breadcrumb": [
            "properties",
            "likes_getting_petted"
          ]
        }
      ],
      "stream": "animals"
    }
  ]
}

I have added selected flags.

On following command

$ tap-mysql -c config.json --properties properties.json

I have received the following response

{"type": "STATE", "value": {"currently_syncing": null}}

Though my table has rows

Edgar Ramírez Mondragón
  • 2,912
  • 3
  • 19
  • 34
Parithi
  • 51
  • 3

1 Answers1

3

You need to make sure that you are marking that table as 'selected' in your properties.json file. Also make sure you specify the replication-method type.

this section below needs to be changed from

 "metadata": {
     "database-name": "example_db",
     "selected-by-default": false,
     "is-view": false,
     "row-count": 3
 },
 "breadcrumb": []

to

 "metadata": {
     "database-name": "example_db",
     "selected-by-default": false,
     "is-view": false,
     "row-count": 3,
     "selected": true,
     "replication-method": "FULL_TABLE"
 },
 "breadcrumb": []

The two pieces I believe you are missing are the two lines below:

"selected": true,
"replication-method": "FULL_TABLE"

See the GitHub documentation example for further clarification: https://github.com/singer-io/tap-mysql#replication-methods-and-state-file

log4code
  • 133
  • 1
  • 7