Solr DataImportHandler configuration

Question 1

Solr DataImportHandler configuration

mysql solr indexing dataimporthandler requesthandler

talo · Jun 19, 2012 · Viewed 7.2k times · Source

Answer

Answer

The issue here is your "lib" entries with jar file location is not properly defined.

Depending on your solr.home entry, these dist, contrib folders are not resolving. For instance, you may need to additional "../" for this..

Question 2

I want to get data from mysql database with the help of DataImportHandler so i can create indexes. Now I've configured my Solr instance so that it works on Tomcat (the example admin page), but if I try to change the sorlconfig.xml file i'll get the error message. I'm working with Solr 3.6

So my configuration is:

In solrconfig.xml I added:

<dataDir>${solr.data.dir:/usr/share/tomcat7/solr2}</dataDir>

to specify my working directory

and then

<requestHandler name="/dataimport" class="org.apache.solr.handler.dataimport.DataImportHandler">
<lst name="defaults">
<str name="config">/usr/share/tomcat7/solr2/conf/data-config.xml</str> 
</lst>
</requestHandler>

to specify new request handler.

Theese two are my lib directives for DIH. Do i need to change them?

<lib dir="../../dist/" regex="apache-solr-dataimporthandler-\d.*\.jar" />
<lib dir="../../contrib/dataimporthandler/lib/" regex=".*\.jar" />

I also created data-config.xml file and added following:

<?xml version="1.0" encoding="UTF-8"?>
<dataConfig>
<dataSource   type="JdbcDataSource"
              driver="com.mysql.jdbc.Driver"
              url="jdbc:mysql://localhost:3306/ethicsweb_experts"
              user="root"
              password=""/>

<document>
       <entity name="experts" datasource="mysql" pk="mainid"
              query="SELECT experts.mainid as mainid FROM experts WHERE validRec = 'y'">

        <field column="mainid" name="mainid"/>
        <field column="validRec" name="validRec"/>
    </entity>
</document>

I've coppied following jars to tomcat/lib folder (DIH jar files and mysql JDBC connector jar file)

apache-solr-dataimporthandler-3.6.0.jar
apache-solr-dataimporthandler-extras-3.6.0.jar
mysql-connector-java-5.1.20-bin.jar

Also in the schema.xml file i added folowing fields:

<field name="mainid" type="int" indexed="true" stored="true"  />
<field name="validRec" type="string" indexed="true" stored="true" />
<field name="recSource" type="string" indexed="true" stored="true" />

<uniqueKey>mainid</uniqueKey>

But now when i try to acces:

http://localhost:8080/solr2/

I get following output:

HTTP Status 500 - Severe errors in solr configuration. Check your log files for more detailed information on what may be wrong. If you want solr to continue after configuration errors, change: <abortOnConfigurationError>false</abortOnConfigurationError> in solr.xml ------------------------------------------------------------- 
org.apache.solr.common.SolrException: No cores were created, please check the logs for errors at org.apache.solr.core.CoreContainer$Initializer.initialize(CoreContainer.java:172) 
at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:96) 
at org.apache.catalina.core.ApplicationFilterConfig.initFilter(ApplicationFilterConfig.java:277) 
at org.apache.catalina.core.ApplicationFilterConfig.getFilter(ApplicationFilterConfig.java:258)
at org.apache.catalina.core.ApplicationFilterConfig.setFilterDef(ApplicationFilterConfig.java:382) 
at org.apache.catalina.core.ApplicationFilterConfig.<init>(ApplicationFilterConfig.java:103) 
at org.apache.catalina.core.StandardContext.filterStart(StandardContext.java:4638) 
at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5294)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150) 
at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:895) 
at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:871) 
at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:615) 
at org.apache.catalina.startup.HostConfig.deployDescriptor(HostConfig.java:649) 
at org.apache.catalina.startup.HostConfig$DeployDescriptor.run(HostConfig.java:1585) 
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) 
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) 
at java.util.concurrent.FutureTask.run(FutureTask.java:166) 
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) 
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) 
at java.lang.Thread.run(Thread.java:722)
 -------------------------------------------------------------         
java.lang.NoClassDefFoundError: org/apache/solr/util/plugin/SolrCoreAware 
at java.lang.ClassLoader.defineClass1(Native Method) 
at java.lang.ClassLoader.defineClass(ClassLoader.java:791) 
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) 
at java.net.URLClassLoader.defineClass(URLClassLoader.java:449) 
at java.net.URLClassLoader.access$100(URLClassLoader.java:71) 
at java.net.URLClassLoader$1.run(URLClassLoader.java:361) 
at java.net.URLClassLoader$1.run(URLClassLoader.java:355) 
at java.security.AccessController.doPrivileged(Native Method) 
at java.net.URLClassLoader.findClass(URLClassLoader.java:354) 
at java.lang.ClassLoader.loadClass(ClassLoader.java:423) 
at java.lang.ClassLoader.loadClass(ClassLoader.java:356) 
at java.lang.Class.forName0(Native Method) 
at java.lang.Class.forName(Class.java:264) 
at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1698) 
at java.lang.ClassLoader.loadClass(ClassLoader.java:410) 
at java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:789) 
at java.lang.ClassLoader.loadClass(ClassLoader.java:356) 
at java.lang.Class.forName0(Native Method) 
at java.lang.Class.forName(Class.java:264) 
at org.apache.solr.core.SolrResourceLoader.findClass(SolrResourceLoader.java:378) 
at org.apache.solr.core.SolrCore.createInstance(SolrCore.java:419) 
at org.apache.solr.core.SolrCore.createRequestHandler(SolrCore.java:455) 
at org.apache.solr.core.RequestHandlers.initHandlersFromConfig(RequestHandlers.java:159) 
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:563) 
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:483) 
at org.apache.solr.core.CoreContainer.load(CoreContainer.java:335) 
at org.apache.solr.core.CoreContainer.load(CoreContainer.java:219) 
at org.apache.solr.core.CoreContainer$Initializer.initialize(CoreContainer.java:161) 
at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:96) 
at org.apache.catalina.core.ApplicationFilterConfig.initFilter(ApplicationFilterConfig.java:277) 
at org.apache.catalina.core.ApplicationFilterConfig.getFilter(ApplicationFilterConfig.java:258) 
at org.apache.catalina.core.ApplicationFilterConfig.setFilterDef(ApplicationFilterConfig.java:382) 
at org.apache.catalina.core.ApplicationFilterConfig.<init>(ApplicationFilterConfig.java:103) 
at org.apache.catalina.core.StandardContext.filterStart(StandardContext.java:4638) 
at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5294) 
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150) 
at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:895) 
at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:871) 
at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:615) 
at org.apache.catalina.startup.HostConfig.deployDescriptor(HostConfig.java:649) 
at org.apache.catalina.startup.HostConfig$DeployDescriptor.run(HostConfig.java:1585) 
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) 
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) 
at java.util.concurrent.FutureTask.run(FutureTask.java:166) 
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) 
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) 
at java.lang.Thread.run(Thread.java:722) Caused by: java.lang.ClassNotFoundException: org.apache.solr.util.plugin.SolrCoreAware 
at java.net.URLClassLoader$1.run(URLClassLoader.java:366) 
at java.net.URLClassLoader$1.run(URLClassLoader.java:355) 
at java.security.AccessController.doPrivileged(Native Method) 
at java.net.URLClassLoader.findClass(URLClassLoader.java:354) 
at java.lang.ClassLoader.loadClass(ClassLoader.java:423) 
at java.lang.ClassLoader.loadClass(ClassLoader.java:356) ... 47 more

My log entries show me that:

SEVERE: java.lang.NoClassDefFoundError: org/apache/solr/util/plugin/SolrCoreAware
    at java.lang.ClassLoader.defineClass1(Native Method)
    at java.lang.ClassLoader.defineClass(ClassLoader.java:791)
    at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
    at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
    at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:423)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:356)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:264)
    at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1698)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:410)
    at java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:789)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:356)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:264)
    at org.apache.solr.core.SolrResourceLoader.findClass(SolrResourceLoader.java:378)
    at org.apache.solr.core.SolrCore.createInstance(SolrCore.java:419)
    at org.apache.solr.core.SolrCore.createRequestHandler(SolrCore.java:455)
    at org.apache.solr.core.RequestHandlers.initHandlersFromConfig(RequestHandlers.java:159)
    at org.apache.solr.core.SolrCore.<init>(SolrCore.java:563)
    at org.apache.solr.core.CoreContainer.create(CoreContainer.java:483)
    at org.apache.solr.core.CoreContainer.load(CoreContainer.java:335)
    at org.apache.solr.core.CoreContainer.load(CoreContainer.java:219)
    at org.apache.solr.core.CoreContainer$Initializer.initialize(CoreContainer.java:161)
    at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:96)
    at org.apache.catalina.core.ApplicationFilterConfig.initFilter(ApplicationFilterConfig.java:277)
    at org.apache.catalina.core.ApplicationFilterConfig.getFilter(ApplicationFilterConfig.java:258)
    at org.apache.catalina.core.ApplicationFilterConfig.setFilterDef(ApplicationFilterConfig.java:382)
    at org.apache.catalina.core.ApplicationFilterConfig.<init>(ApplicationFilterConfig.java:103)
    at org.apache.catalina.core.StandardContext.filterStart(StandardContext.java:4638)
    at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5294)
    at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
    at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:895)
    at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:871)
    at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:615)
    at org.apache.catalina.startup.HostConfig.deployDescriptor(HostConfig.java:649)
    at org.apache.catalina.startup.HostConfig$DeployDescriptor.run(HostConfig.java:1585)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
    at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
    at java.util.concurrent.FutureTask.run(FutureTask.java:166)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
    at java.lang.Thread.run(Thread.java:722)
Caused by: java.lang.ClassNotFoundException: org.apache.solr.util.plugin.SolrCoreAware
    at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:423)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:356)
    ... 47 more

Jun 15, 2012 4:07:50 PM org.apache.solr.servlet.SolrDispatchFilter init
SEVERE: Could not start Solr. Check solr/home property and the logs
org.apache.solr.common.SolrException: No cores were created, please check the logs for errors
    at org.apache.solr.core.CoreContainer$Initializer.initialize(CoreContainer.java:172)
    at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:96)
    at org.apache.catalina.core.ApplicationFilterConfig.initFilter(ApplicationFilterConfig.java:277)
    at org.apache.catalina.core.ApplicationFilterConfig.getFilter(ApplicationFilterConfig.java:258)
    at org.apache.catalina.core.ApplicationFilterConfig.setFilterDef(ApplicationFilterConfig.java:382)
    at org.apache.catalina.core.ApplicationFilterConfig.<init>(ApplicationFilterConfig.java:103)
    at org.apache.catalina.core.StandardContext.filterStart(StandardContext.java:4638)
    at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5294)
    at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
    at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:895)
    at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:871)
    at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:615)
    at org.apache.catalina.startup.HostConfig.deployDescriptor(HostConfig.java:649)
    at org.apache.catalina.startup.HostConfig$DeployDescriptor.run(HostConfig.java:1585)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
    at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
    at java.util.concurrent.FutureTask.run(FutureTask.java:166)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
    at java.lang.Thread.run(Thread.java:722)

Jun 15, 2012 4:07:50 PM org.apache.solr.common.SolrException log
SEVERE: org.apache.solr.common.SolrException: No cores were created, please check the logs for errors
    at org.apache.solr.core.CoreContainer$Initializer.initialize(CoreContainer.java:172)
    at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:96)
    at org.apache.catalina.core.ApplicationFilterConfig.initFilter(ApplicationFilterConfig.java:277)
    at org.apache.catalina.core.ApplicationFilterConfig.getFilter(ApplicationFilterConfig.java:258)
    at org.apache.catalina.core.ApplicationFilterConfig.setFilterDef(ApplicationFilterConfig.java:382)
    at org.apache.catalina.core.ApplicationFilterConfig.<init>(ApplicationFilterConfig.java:103)
    at org.apache.catalina.core.StandardContext.filterStart(StandardContext.java:4638)
    at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5294)
    at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
    at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:895)
    at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:871)
    at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:615)
    at org.apache.catalina.startup.HostConfig.deployDescriptor(HostConfig.java:649)
    at org.apache.catalina.startup.HostConfig$DeployDescriptor.run(HostConfig.java:1585)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
    at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
    at java.util.concurrent.FutureTask.run(FutureTask.java:166)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
    at java.lang.Thread.run(Thread.java:722)

So now I wonder where did I screw up the configuration for the DataImportHandler. Do I need to specify more jar files? Did I put jar files in the correct directory? Any help would be greatly appreciated.

Solr DataImportHandler configuration

Answer

Related questions