Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add "Evaluating accuracy" section #752

Open
wants to merge 5 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -234,6 +234,15 @@ To use your webcam/camera, simply replace `videofile.avi` with keyword `camera`.

To save a video with predicted bounding box, add `--saveVideo` option.

## Evaluating accuracy
To evaluate the performance of your model, you can use **mean Average Precision (mAP)** metric from [this repository here](https://github.com/Cartucho/mAP). It works well with darkflow's predicted `json` files.

1) Clone [mAP](https://github.com/Cartucho/mAP) repository
2) Flow your darkflow model with `--json` option. Then move the predicted `json` files into the `predicted` folder of mAP repo.
3) Move your ground truth annotation files into `ground-truth` folder. (E.g. `.xml` files generated from [labelImg](https://github.com/tzutalin/labelImg) or `.txt` files generated from [BBox-Label-Tool](https://github.com/puzzledqs/BBox-Label-Tool))
4) Read the `README.md` and `extra/README.md` of that repository for further information, you'll need to run some `extra/` scripts based on your circumstance.
5) Run `python main.py` of mAP repo.

## Using darkflow from another python application

Please note that `return_predict(img)` must take an `numpy.ndarray`. Your image must be loaded beforehand and passed to `return_predict(img)`. Passing the file path won't work.
Expand Down
8 changes: 6 additions & 2 deletions darkflow/utils/pascal_voc_clean_xml.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,8 @@ def pascal_voc_clean_xml(ANN, pick, exclusive = False):
cur_dir = os.getcwd()
os.chdir(ANN)
annotations = os.listdir('.')
annotations = glob.glob(str(annotations)+'*.xml')
# annotations = glob.glob(str(annotations)+'*.xml')
annotations = glob.glob('*.xml')
size = len(annotations)

for i, file in enumerate(annotations):
Expand All @@ -36,7 +37,10 @@ def pascal_voc_clean_xml(ANN, pick, exclusive = False):
in_file = open(file)
tree=ET.parse(in_file)
root = tree.getroot()
jpg = str(root.find('filename').text)
# jpg = str(root.find('filename').text)
from pathlib import Path
jpg = str(Path(file).with_suffix('.jpg').name)
# print('jpg:', jpg)
imsize = root.find('size')
w = int(imsize.find('width').text)
h = int(imsize.find('height').text)
Expand Down
Binary file modified preview.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified sample_img/sample_computer.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified sample_img/sample_dog.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified sample_img/sample_eagle.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified sample_img/sample_giraffe.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified sample_img/sample_horses.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified sample_img/sample_person.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified sample_img/sample_scream.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified test/training/images/1.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified test/training/images/2.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.