Importing MySQL databases using Solr DataImportHandler

Teguh Arief

Sep 24, 2020

Share article to

Illustration of importing MySQL databases using Solr.

Solr is both a search engine and a distributed document database. In Solr method, the DataImportHandler provides a mechanism for importing content from a data store and indexing it. The main advantage of this method of data importing is no need for additional software development and the rapid integration of the data source.

Table of structure:




  • virtual-host/


    • docker-composer.yml

    • etc/

      • opt/

        • solr/

          • lib/

            • ...

            • mysql-connector-java-5.1.46.jar

            • mysql-connector-java-5.1.46-bin.jar

            • ...

            • solr-dataimporthandler-4.10.1.jar

            • solr-dataimporthandler-extras-4.10.1.jar

            • ...



          • mycore/

            • ...

            • data-config.xml

            • ...

            • schema.xml

            • ...

            • solrconfig.xml

            • ...



          • solr.xml











 



Docker Container




$ docker pull mysql:5.7.19
$ docker pull 2degrees/solr4


 



Setup



docker-compose.yml


version: '2'
services:
mysql:
image: mysql:5.7.19
environment:
- MYSQL_USER=root
- MYSQL_ROOT_PASSWORD=123456
ports:
- "3306:3306"
solr:
image: 2degrees/solr4
volumes:
- "./etc/opt/solr:/etc/opt/solr"
- "./etc/opt/solr/mycore:/etc/opt/solr/mycore"
- "./etc/opt/solr/lib:/etc/opt/solr/lib"
ports:
- "8983:8983"
links:
- mysql


Download Apache Solr DataImportHandler Copy file from the downloaded archive 'solr-dataimporthandler-*.jar' to the folder 'lib'. Download MySQL connector Copy file from the downloaded archive 'mysql-connector-java-*.jar' to the folder 'lib'. solr.xml Create lines to 'solr.xml':










solrconfig.xml Edit file 'solrconfig.xml' by adding:



 


And add these lines to 'solrconfig.xml':




data-config.xml




data-config.xml for MySQL database the file 'data-config.xml' will define data to import from a data store.



 


schema.xml Edit file 'schema.xml' accordingly to fields defined in data-config.xml:



 


 



Execute



Now a Full Import operation can be started in Solr screen or with a URL such as http://192.168.99.100:8983/solr/mycore/dataimport?command=full-import Solr shows the configuration of the DataImportHandler and allows you to start, and monitor the status of, import commands as defined by the options selected on the Solr screen and defined in the configuration file.