Support ES6 - collections go in their own index option.

[thejeff77]

Simply add an option to put collections into their own named index, instead of all in the db name. This would support the main breaking feature of ES6, as it is working fine for me otherwise.

[a-magdy]

Hey @thejeff77

I think you can do it yourself by renaming each collection to its own index:

https://github.com/mongodb-labs/mongo-connector/wiki/Configuration-Options#renaming-namespaces

The config has changed in mongo-connector v2.5, it is all stated there

In you config file you can send specific collection using the namespaces attribute in your config file

  "namespaces": {
    "db_name.collection1": true,
    "db_name.collection2": true
  }

And you can rename them:

  "namespaces": {
    "db_name.collection1": {
      "rename": "collection1._doc"
    },
    "db_name.collection2": {
      "rename": "collection2._doc"
    }
  }

I hope it helps

---

Update 1:

I tried it locally with ES5 & ES6 .. my observation is that with ES5, using collection1._doc didn't work, but collection1.doc works (ES6 accepts _doc but ES5 doesn't -had to be doc with ES5-)

[dikshitluthra]

My database name and collection names are dynamic. Is there a way to handle this? Can you point me the file with relevant code so that I can fork and make required changes?

[mshzsh]

hi dear @a-magdy

i can't find & access config.json file!
please explain clearly

[a-magdy]

Hi @mshzsh

You create your own config.json file and fill it with the information about the connection by following this documentation: https://github.com/yougov/mongo-connector/wiki/Configuration-Options

You can do some stuff with the command line

Like:

{
  "mainAddress": "mongodb://...",
  "oplogFile": "<path-to-oplog-file>",
  "noDump": false,
  ...,
  ...
}

And in that file, you can add more configuration (check the docs in the link sent above for what can be done)

So in this case if you want to rename your collections to be in their own index in Elasticsearch, you add similar lines:

{
  "mainAddress": "mongodb://...",
  "oplogFile": "<path-to-oplog-file>",
  "noDump": false,
  ...,
  ...
  "namespaces": {
    "db_name.collection1": {
      "rename": "collection1._doc"
    },
    "db_name.collection2": {
      "rename": "collection2._doc"
    }
  }
}

Then you start mongo-connector by passing the file path as an argument: mongo-connector -c config.json

I hope that helps

P.S I actually switched at some point to another tool called monstache it is better supported, and has more features, you should check it out

[mshzsh]

@a-magdy thank you!

[tigerking]

pls check if it works for you #60