Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Qa updates #374

Open
wants to merge 2 commits into
base: QuestionAnswering
Choose a base branch
from
Open

Qa updates #374

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
c5930d6
New datasets, 1 new QA System, rename QAnswer
LukasBluebaum Oct 30, 2020
af5c702
Comment out not reachable systems
LukasBluebaum Dec 8, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 8 additions & 3 deletions pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -313,7 +313,7 @@
<dependency>
<groupId>org.aksw.qa</groupId>
<artifactId>systems</artifactId>
<version>0.0.6-SNAPSHOT</version>
<version>0.0.7-SNAPSHOT</version>
<exclusions>
<exclusion>
<groupId>org.aksw.qa</groupId>
Expand All @@ -340,7 +340,7 @@
<dependency>
<groupId>org.aksw.qa</groupId>
<artifactId>commons</artifactId>
<version>0.4.22</version>
<version>0.4.23</version>
<exclusions>
<exclusion>
<groupId>ch.qos.logback</groupId>
Expand Down Expand Up @@ -400,7 +400,7 @@
<dependency>
<groupId>org.aksw.qa</groupId>
<artifactId>datasets</artifactId>
<version>0.5.17</version>
<version>0.5.18</version>
</dependency>
<!-- https://mvnrepository.com/artifact/com.github.jsonld-java/jsonld-java -->
<dependency>
Expand All @@ -425,6 +425,11 @@
<artifactId>jena-core</artifactId>
<version>3.6.0</version>
</dependency>
<dependency>
<groupId>org.reflections</groupId>
<artifactId>reflections</artifactId>
<version>0.9.12</version>
</dependency>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this dependency needed? The project is built for me without issues when I comment out the dependency 🤔

Copy link
Contributor Author

@LukasBluebaum LukasBluebaum Dec 20, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for the late answer.
The thing is when I don't set the version to 0.9.12, then version 0.9.11 will be used. And that version seems to have a multithreading issue.

So in the default case in NLIWODBasedSystem the reflections dependency is used to retrieve names of systems in qa.systems. Now, when an experiment is started with multiple systems that execute the default case (e.g. TeBaQA, QAmp, Platypus), then two of the threads will throw a java.lang.IllegalStateException: zip file closed exception. Version 0.9.12 doesn't seem to have this issue.

</dependencies>

<build>
Expand Down
4 changes: 2 additions & 2 deletions src/main/java/org/aksw/gerbil/annotator/impl/qa/NLIWODBasedSystem.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,8 +38,8 @@ public class NLIWODBasedSystem extends AbstractAnnotator implements QASystem {
public static final String START_SYSTEM_NAME = "START";
public static final String YODA_SYSTEM_NAME = "YODA";
public static final String OKBQA_SYSTEM_NAME = "OKBQA";
public static final String QANARY_SYSTEM_WIKIDATA_NAME = "wdaqua-core1 (wikidata)";
public static final String QANARY_SYSTEM_DBPEDIA_NAME = "wdaqua-core1 (DBpedia)";
public static final String QANARY_SYSTEM_WIKIDATA_NAME = "QAnswer (wikidata)";
public static final String QANARY_SYSTEM_DBPEDIA_NAME = "QAnswer (DBpedia)";
public static final String GANSWER2_SYSTEM_NAME = "gAnswer2";
public static final String QUEPY = "QUEPY";
private static final int DEFAULT_WAITING_TIME = 60000;
Expand Down
74 changes: 40 additions & 34 deletions src/main/properties/annotators.properties
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -157,18 +157,18 @@ org.aksw.gerbil.annotators.definition.NERD.constructorArgs=${org.aksw.gerbil.ann
#org.aksw.gerbil.annotators.definition.HAWK.class=org.aksw.gerbil.annotator.impl.qa.NLIWODBasedSystem
#org.aksw.gerbil.annotators.definition.HAWK.constructorArgs=HAWK
#org.aksw.gerbil.annotators.definition.HAWK.singleton=true
org.aksw.gerbil.annotators.definition.QAKIS.name=QAKIS
org.aksw.gerbil.annotators.definition.QAKIS.experimentType=QA
org.aksw.gerbil.annotators.definition.QAKIS.cacheable=true
org.aksw.gerbil.annotators.definition.QAKIS.class=org.aksw.gerbil.annotator.impl.qa.NLIWODBasedSystem
org.aksw.gerbil.annotators.definition.QAKIS.constructorArgs=QAKIS
org.aksw.gerbil.annotators.definition.QAKIS.singleton=true
org.aksw.gerbil.annotators.definition.SINA.name=SINA
org.aksw.gerbil.annotators.definition.SINA.experimentType=QA
org.aksw.gerbil.annotators.definition.SINA.cacheable=true
org.aksw.gerbil.annotators.definition.SINA.class=org.aksw.gerbil.annotator.impl.qa.NLIWODBasedSystem
org.aksw.gerbil.annotators.definition.SINA.constructorArgs=SINA
org.aksw.gerbil.annotators.definition.SINA.singleton=true
#org.aksw.gerbil.annotators.definition.QAKIS.name=QAKIS
#org.aksw.gerbil.annotators.definition.QAKIS.experimentType=QA
#org.aksw.gerbil.annotators.definition.QAKIS.cacheable=true
#org.aksw.gerbil.annotators.definition.QAKIS.class=org.aksw.gerbil.annotator.impl.qa.NLIWODBasedSystem
#org.aksw.gerbil.annotators.definition.QAKIS.constructorArgs=QAKIS
#org.aksw.gerbil.annotators.definition.QAKIS.singleton=true
#org.aksw.gerbil.annotators.definition.SINA.name=SINA
#org.aksw.gerbil.annotators.definition.SINA.experimentType=QA
#org.aksw.gerbil.annotators.definition.SINA.cacheable=true
#org.aksw.gerbil.annotators.definition.SINA.class=org.aksw.gerbil.annotator.impl.qa.NLIWODBasedSystem
#org.aksw.gerbil.annotators.definition.SINA.constructorArgs=SINA
#org.aksw.gerbil.annotators.definition.SINA.singleton=true
#org.aksw.gerbil.annotators.definition.START.name=START
#org.aksw.gerbil.annotators.definition.START.experimentType=QA
#org.aksw.gerbil.annotators.definition.START.cacheable=true
Expand All @@ -181,17 +181,17 @@ org.aksw.gerbil.annotators.definition.SINA.singleton=true
#org.aksw.gerbil.annotators.definition.YODA.class=org.aksw.gerbil.annotator.impl.qa.NLIWODBasedSystem
#org.aksw.gerbil.annotators.definition.YODA.constructorArgs=YODA
#org.aksw.gerbil.annotators.definition.YODA.singleton=true
org.aksw.gerbil.annotators.definition.QANARYwiki.name=wdaqua-core1 (wikidata)
org.aksw.gerbil.annotators.definition.QANARYwiki.name=QAnswer (wikidata)
org.aksw.gerbil.annotators.definition.QANARYwiki.experimentType=QA
org.aksw.gerbil.annotators.definition.QANARYwiki.cacheable=true
org.aksw.gerbil.annotators.definition.QANARYwiki.class=org.aksw.gerbil.annotator.impl.qa.NLIWODBasedSystem
org.aksw.gerbil.annotators.definition.QANARYwiki.constructorArgs=wdaqua-core1 (wikidata), http://qanswer-core1.univ-st-etienne.fr/gerbil
org.aksw.gerbil.annotators.definition.QANARYwiki.constructorArgs=QAnswer (wikidata), http://qanswer-core1.univ-st-etienne.fr/api/gerbil
org.aksw.gerbil.annotators.definition.QANARYwiki.singleton=true
org.aksw.gerbil.annotators.definition.QANARYdbpedia.name=wdaqua-core1 (DBpedia)
org.aksw.gerbil.annotators.definition.QANARYdbpedia.name=QAnswer (DBpedia)
org.aksw.gerbil.annotators.definition.QANARYdbpedia.experimentType=QA
org.aksw.gerbil.annotators.definition.QANARYdbpedia.cacheable=true
org.aksw.gerbil.annotators.definition.QANARYdbpedia.class=org.aksw.gerbil.annotator.impl.qa.NLIWODBasedSystem
org.aksw.gerbil.annotators.definition.QANARYdbpedia.constructorArgs=wdaqua-core1 (DBpedia), http://qanswer-core1.univ-st-etienne.fr/gerbil
org.aksw.gerbil.annotators.definition.QANARYdbpedia.constructorArgs=QAnswer (DBpedia), http://qanswer-core1.univ-st-etienne.fr/api/gerbil
org.aksw.gerbil.annotators.definition.QANARYdbpedia.singleton=true
org.aksw.gerbil.annotators.definition.OKBQA.name=OKBQA
org.aksw.gerbil.annotators.definition.OKBQA.experimentType=QA
Expand All @@ -205,12 +205,12 @@ org.aksw.gerbil.annotators.definition.GANSWER2.cacheable=true
org.aksw.gerbil.annotators.definition.GANSWER2.class=org.aksw.gerbil.annotator.impl.qa.NLIWODBasedSystem
org.aksw.gerbil.annotators.definition.GANSWER2.constructorArgs=gAnswer2, http://59.108.48.18:8080/gAnswer2/api/qald.jsp
org.aksw.gerbil.annotators.definition.GANSWER2.singleton=true
org.aksw.gerbil.annotators.definition.QUEPY.name=QUEPY
org.aksw.gerbil.annotators.definition.QUEPY.experimentType=QA
org.aksw.gerbil.annotators.definition.QUEPY.cacheable=true
org.aksw.gerbil.annotators.definition.QUEPY.class=org.aksw.gerbil.annotator.impl.qa.NLIWODBasedSystem
org.aksw.gerbil.annotators.definition.QUEPY.constructorArgs=QUEPY, http://quepy.machinalis.com/engine/get_query
org.aksw.gerbil.annotators.definition.QUEPY.singleton=true
#org.aksw.gerbil.annotators.definition.QUEPY.name=QUEPY
#org.aksw.gerbil.annotators.definition.QUEPY.experimentType=QA
#org.aksw.gerbil.annotators.definition.QUEPY.cacheable=true
#org.aksw.gerbil.annotators.definition.QUEPY.class=org.aksw.gerbil.annotator.impl.qa.NLIWODBasedSystem
#org.aksw.gerbil.annotators.definition.QUEPY.constructorArgs=QUEPY, http://quepy.machinalis.com/engine/get_query
#org.aksw.gerbil.annotators.definition.QUEPY.singleton=true
org.aksw.gerbil.annotators.definition.Platypus.name=Platypus
org.aksw.gerbil.annotators.definition.Platypus.experimentType=QA
org.aksw.gerbil.annotators.definition.Platypus.cacheable=true
Expand All @@ -223,18 +223,24 @@ org.aksw.gerbil.annotators.definition.TeBaQA.cacheable=true
org.aksw.gerbil.annotators.definition.TeBaQA.class=org.aksw.gerbil.annotator.impl.qa.NLIWODBasedSystem
org.aksw.gerbil.annotators.definition.TeBaQA.constructorArgs=TeBaQA
org.aksw.gerbil.annotators.definition.TeBaQA.singleton=true
org.aksw.gerbil.annotators.definition.AskNow.name=AskNow
org.aksw.gerbil.annotators.definition.AskNow.experimentType=QA
org.aksw.gerbil.annotators.definition.AskNow.cacheable=true
org.aksw.gerbil.annotators.definition.AskNow.class=org.aksw.gerbil.annotator.impl.qa.NLIWODBasedSystem
org.aksw.gerbil.annotators.definition.AskNow.constructorArgs=AskNow
org.aksw.gerbil.annotators.definition.AskNow.singleton=true
org.aksw.gerbil.annotators.definition.SorokinQA.name=SorokinQA
org.aksw.gerbil.annotators.definition.SorokinQA.experimentType=QA
org.aksw.gerbil.annotators.definition.SorokinQA.cacheable=true
org.aksw.gerbil.annotators.definition.SorokinQA.class=org.aksw.gerbil.annotator.impl.qa.NLIWODBasedSystem
org.aksw.gerbil.annotators.definition.SorokinQA.constructorArgs=SorokinQA
org.aksw.gerbil.annotators.definition.SorokinQA.singleton=true
#org.aksw.gerbil.annotators.definition.AskNow.name=AskNow
#org.aksw.gerbil.annotators.definition.AskNow.experimentType=QA
#org.aksw.gerbil.annotators.definition.AskNow.cacheable=true
#org.aksw.gerbil.annotators.definition.AskNow.class=org.aksw.gerbil.annotator.impl.qa.NLIWODBasedSystem
#org.aksw.gerbil.annotators.definition.AskNow.constructorArgs=AskNow
#org.aksw.gerbil.annotators.definition.AskNow.singleton=true
#org.aksw.gerbil.annotators.definition.SorokinQA.name=SorokinQA
#org.aksw.gerbil.annotators.definition.SorokinQA.experimentType=QA
#org.aksw.gerbil.annotators.definition.SorokinQA.cacheable=true
#org.aksw.gerbil.annotators.definition.SorokinQA.class=org.aksw.gerbil.annotator.impl.qa.NLIWODBasedSystem
#org.aksw.gerbil.annotators.definition.SorokinQA.constructorArgs=SorokinQA
#org.aksw.gerbil.annotators.definition.SorokinQA.singleton=true
org.aksw.gerbil.annotators.definition.QAmp.name=QAmp
org.aksw.gerbil.annotators.definition.QAmp.experimentType=QA
org.aksw.gerbil.annotators.definition.QAmp.cacheable=true
org.aksw.gerbil.annotators.definition.QAmp.class=org.aksw.gerbil.annotator.impl.qa.NLIWODBasedSystem
org.aksw.gerbil.annotators.definition.QAmp.constructorArgs=QAmp
org.aksw.gerbil.annotators.definition.QAmp.singleton=true


### Tagme
Expand Down
50 changes: 48 additions & 2 deletions src/main/properties/datasets.properties
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -120,7 +120,7 @@ org.aksw.gerbil.datasets.definition.Micro2013_2.experimentType=A2KB
org.aksw.gerbil.datasets.definition.Micro2013_2.constructorArgs=${org.aksw.gerbil.datasets.Microposts2013DatasetConfig.test}


### N� collection
### N� collection
org.aksw.gerbil.datasets.N3_NEWS_100.file=${org.aksw.gerbil.DataPath}/datasets/N3/News-100.ttl
org.aksw.gerbil.datasets.N3_REUTERS_128.file=${org.aksw.gerbil.DataPath}/datasets/N3/Reuters-128.ttl
org.aksw.gerbil.datasets.N3_RSS_500.file=${org.aksw.gerbil.DataPath}/datasets/N3/RSS-500.ttl
Expand Down Expand Up @@ -376,6 +376,18 @@ org.aksw.gerbil.datasets.definition.LCQUAD.cacheable=true
org.aksw.gerbil.datasets.definition.LCQUAD.experimentType=QA
org.aksw.gerbil.datasets.definition.LCQUAD.constructorArgs=LCQUAD

org.aksw.gerbil.datasets.definition.LCQUAD2_Train.name=LCQUAD2 Train
org.aksw.gerbil.datasets.definition.LCQUAD2_Train.class=org.aksw.gerbil.dataset.impl.qald.QALDDataset
org.aksw.gerbil.datasets.definition.LCQUAD2_Train.cacheable=true
org.aksw.gerbil.datasets.definition.LCQUAD2_Train.experimentType=QA
org.aksw.gerbil.datasets.definition.LCQUAD2_Train.constructorArgs=LCQUAD2_Train

org.aksw.gerbil.datasets.definition.LCQUAD2_Test.name=LCQUAD2 Test
org.aksw.gerbil.datasets.definition.LCQUAD2_Test.class=org.aksw.gerbil.dataset.impl.qald.QALDDataset
org.aksw.gerbil.datasets.definition.LCQUAD2_Test.cacheable=true
org.aksw.gerbil.datasets.definition.LCQUAD2_Test.experimentType=QA
org.aksw.gerbil.datasets.definition.LCQUAD2_Test.constructorArgs=LCQUAD2_Test

org.aksw.gerbil.datasets.definition.SemSearch.name=DBpedia Entity SemSearch
org.aksw.gerbil.datasets.definition.SemSearch.class=org.aksw.gerbil.dataset.impl.qald.QALDDataset
org.aksw.gerbil.datasets.definition.SemSearch.cacheable=true
Expand Down Expand Up @@ -414,4 +426,38 @@ org.aksw.gerbil.datasets.definition.QALD9_Test_Multilingual.cacheable=true
org.aksw.gerbil.datasets.definition.QALD9_Test_Multilingual.experimentType=QA
org.aksw.gerbil.datasets.definition.QALD9_Test_Multilingual.constructorArgs=QALD9_Test_Multilingual


org.aksw.gerbil.datasets.definition.SimpleQuestions_Train.name=SimpleQuestionsWikidata Train
org.aksw.gerbil.datasets.definition.SimpleQuestions_Train.class=org.aksw.gerbil.dataset.impl.qald.QALDDataset
org.aksw.gerbil.datasets.definition.SimpleQuestions_Train.cacheable=true
org.aksw.gerbil.datasets.definition.SimpleQuestions_Train.experimentType=QA
org.aksw.gerbil.datasets.definition.SimpleQuestions_Train.constructorArgs=Simple_Question_Wikidata_Train

org.aksw.gerbil.datasets.definition.SimpleQuestions_Test.name=SimpleQuestionsWikidata Test
org.aksw.gerbil.datasets.definition.SimpleQuestions_Test.class=org.aksw.gerbil.dataset.impl.qald.QALDDataset
org.aksw.gerbil.datasets.definition.SimpleQuestions_Test.cacheable=true
org.aksw.gerbil.datasets.definition.SimpleQuestions_Test.experimentType=QA
org.aksw.gerbil.datasets.definition.SimpleQuestions_Test.constructorArgs=Simple_Question_Wikidata_Test

org.aksw.gerbil.datasets.definition.SimpleQuestions_Valid.name=SimpleQuestionsWikidata Valid
org.aksw.gerbil.datasets.definition.SimpleQuestions_Valid.class=org.aksw.gerbil.dataset.impl.qald.QALDDataset
org.aksw.gerbil.datasets.definition.SimpleQuestions_Valid.cacheable=true
org.aksw.gerbil.datasets.definition.SimpleQuestions_Valid.experimentType=QA
org.aksw.gerbil.datasets.definition.SimpleQuestions_Valid.constructorArgs=Simple_Question_Wikidata_Valid

org.aksw.gerbil.datasets.definition.EventQA_DBpedia.name=EventQA (DBpedia)
org.aksw.gerbil.datasets.definition.EventQA_DBpedia.class=org.aksw.gerbil.dataset.impl.qald.QALDDataset
org.aksw.gerbil.datasets.definition.EventQA_DBpedia.cacheable=true
org.aksw.gerbil.datasets.definition.EventQA_DBpedia.experimentType=QA
org.aksw.gerbil.datasets.definition.EventQA_DBpedia.constructorArgs=EventQA_DBpedia

org.aksw.gerbil.datasets.definition.EventQA_EventKG.name=EventQA (EventKG)
org.aksw.gerbil.datasets.definition.EventQA_EventKG.class=org.aksw.gerbil.dataset.impl.qald.QALDDataset
org.aksw.gerbil.datasets.definition.EventQA_EventKG.cacheable=true
org.aksw.gerbil.datasets.definition.EventQA_EventKG.experimentType=QA
org.aksw.gerbil.datasets.definition.EventQA_EventKG.constructorArgs=EventQA_EventKG

org.aksw.gerbil.datasets.definition.Wdaqua_core0_logs.name=WDAquaCore0Logs
org.aksw.gerbil.datasets.definition.Wdaqua_core0_logs.class=org.aksw.gerbil.dataset.impl.qald.QALDDataset
org.aksw.gerbil.datasets.definition.Wdaqua_core0_logs.cacheable=true
org.aksw.gerbil.datasets.definition.Wdaqua_core0_logs.experimentType=QA
org.aksw.gerbil.datasets.definition.Wdaqua_core0_logs.constructorArgs=Wdaqua_Core0_Logs
8 changes: 8 additions & 0 deletions src/main/resources/spring/database/schema/experiment-schema.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,3 +50,11 @@ subTaskId int NOT NULL,
PRIMARY KEY (taskId, subTaskId)
);

-- Rename wdaqua-core1 dataset
UPDATE ExperimentTasks
SET annotatorName = 'QAnswer (wikidata)'
WHERE annotatorName = 'wdaqua-core1 (wikidata)';

UPDATE ExperimentTasks
SET annotatorName = 'QAnswer (DBpedia)'
WHERE annotatorName = 'wdaqua-core1 (DBpedia)';