[WIP] Download annotated notebooks #34

peerdavid · 2018-12-04T18:41:32Z

This allows to execute geta (read annotated file) and mgeta (read multiple annotated files)
Note: With this PR we introduce additional dependencies (the transformation is executed i.e. in a bash script). Those are listed in README.md

ToDo's

Set modification date of pdf file and download only if content changed
Bug: Non annotated pdf files are not downloaded correctly
Bug: Ignore epub files
Add function to validate if all tools are installed to execute geta, mgeta
Implement rM2svg instead of calling a script
Implement exportAnnotatedPdf instead of calling a script
Bug: Annotations of rotated pdf files are not rotated

juruen · 2018-12-08T13:19:16Z

@peerdavid 👋 thank you for the PR!

I don't have much time this week to look at this but my proposal would be something along these lines:

We merge this into master but we sort of make sure that this is a an "advanced feature". In the sense that we are clear that you don't need to install all these dependencies if you don't really need to get the annotated PDF docs.
We slowly start to to migrate the scripts to pure Go to simplify the dependencies. No rush here. I'll probably have some time during the holidays. And of course, you are more than welcome to help with that!

What do you think?

peerdavid · 2018-12-08T13:37:48Z

@juruen

Sounds good to me. Possible we can also add a check if all tools are installed and warn the user that this feature is not available when geta / mgeta is called or something like that?
I also think that it totally makes sense to replace the exportAnnotatedPdf and the rM2svg script. In the holidays I should also have some time and of course I will help with that :)

juruen

Thank so much for doing this!

I added a few minor comments.

I would also like to change the README.md to be more explicit about this being optional. As I said in the issue, I don't want to discourage non-power users by letting them think they need to install a lot of dependencies. I'm going to propose some changes to your PR in a separate one to see if you are cool with them.

Thank you!

juruen · 2018-12-23T11:16:03Z

shell/geta.go

+	if err != nil {
+		// If we could not parse the time correctly, we still continue 
+		// with the execution such that the pdf is downloaded...
+		fmt.Println(err)


Could you please use the logging facilities instead of standard output/err?

Something like:

log.Trace.Println(err)

or in this case, if this is just a warning:

log.Warning.Println(err)

In general, we should try to use the log facilities instead of fmt.Print as much as we can in this file. Either that or use the ishell context to print relevant stuff.

juruen · 2018-12-23T11:16:23Z

shell/geta.go

+		// If we could not parse the time correctly, we still continue 
+		// with the execution such that the pdf is downloaded...
+		fmt.Println(err)
+		fmt.Println("(Warning) Could not parse modified time. Overwrite existing file.")


juruen · 2018-12-23T11:16:50Z

shell/geta.go

+	if !os.IsNotExist(err) {
+		outputFileModTime := outputFile.ModTime()
+		if(outputFileModTime.Equal(modifiedClientTime)){
+			fmt.Println("Nothing changed since last download. Skip. ")


Same here. This looks like log.Trace candidate.

Thanks for your comments => fixed all invalid log statements

juruen · 2018-12-23T11:22:42Z

shell/geta.go

+
+	// Convert to pdf
+	exportPdf := os.Getenv("GOPATH") + "/src/github.com/juruen/rmapi/tools/exportAnnotatedPdf"
+	rM2svg := os.Getenv("GOPATH") + "/src/github.com/juruen/rmapi/tools/rM2svg"


How about we create a file/function where we encapsulate checks to verify if the required tools are installed. We can then use it in https://github.com/juruen/rmapi/blob/master/shell/shell.go#L54 to optinally add these new commands to the shell.

Sounds good. I will fix a bug that I found right now and afterwards I will start with this verification

…re disabled 2. Updated README.md to be more explicit about the optional commands geta and mgeta

juruen · 2019-01-01T17:35:36Z

README.md

+
+
+# Thanks to
+[1] rmapi, https://github.com/juruen/rmapi <br />


We can drop this line :)

mseri · 2019-04-11T08:09:30Z

Does this support the latest remarkable line format? The export notebook can be made platform agnostic as done here: reHackable/maxio#22 (and probably it could be ported to go in a similar fashion)

peerdavid · 2019-04-11T09:05:13Z

Does this support the latest remarkable line format? The export notebook can be made platform agnostic as done here: reHackable/maxio#22 (and probably it could be ported to go in a similar fashion)

Currently not but I will update to the latest line format (in the fork that I created) when my RM device is also running on version 1.7

lobre · 2019-04-11T09:06:49Z

Hello @mseri,

This normally temporary feature (as it should be shifted to a full go implementation someday) relies as you said on the rM2svg of https://github.com/reHackable/maxio.

Effectively, it seems to be working with the previous .lines version (see the header here)

Apparently, there is an open issue on the project here.

So I think the implementation has to be made there.

However, we started slowly the shifting to a pure go implementation. We created and merged recently the package github.com/juruen/rmapi/encoding/rm that has the responsibility to encode/decode a .rm file (and that supports only the latest v3 format).

See https://github.com/juruen/rmapi/blob/master/encoding/rm/rm.go#L43.

I am not sure whether it makes sense to continue supporting the previous format as well.

@peerdavid, just seen you reply and did not know that you started a fork with the support of the new format! Good news!

mseri · 2019-04-11T14:23:19Z

@lobre there is already a version working with the new format (see the comments here reHackable/maxio#22 (comment) and here reHackable/maxio#27 (comment), the last version of the script is http://www.ucl.ac.uk/~ucecesf/tmp/rm2svg.py). The https://github.com/reHackable/maxio repo seems unmaintained, there are pending PRs to get rid of external dependencies and other improvements that are ignored since months.

I don't think that supporting the old version makes sense, the recent updates are more stable, more ergonomic and more flexible, I hope people are uploading their remarkables.

martin-braun · 2021-01-28T18:27:50Z

@peerdavid mgeta is not listed when doing rmapi help

peerdavid · 2021-01-28T18:41:34Z

Sry, my initial implementation which uses a python script etc. is no more maintained - they implemented in my opinion a much better golang native version of mgeta which should also be in the masterbranch.

peerdavid added 30 commits October 12, 2018 14:51

Merge remote-tracking branch 'upstream/master'

713b03f

Merge remote-tracking branch 'origin/master'

1660c61

Changes to implement geta

b23b692

Finished geta for svg files

cc06bd9

Finished geta for svg

f967740

Own dir for notebook

01cf2b4

Move into own directory

09d6c6d

Fixed exportNotebook script for rmapi usage

9e48489

Export of annotated pdf

68ebecd

Pdf export for pdf files

4038663

Updated documentation

48f65d1

Change to line2svg

ff515f6

Fix for 1.6 with rM2svg

2417149

Fixed for RM version 1.6

238f38c

Using invisible tmp folders

5d635b3

Test with new rM2svg script

6aea236

Minor changes

d78292c

Fixes for notebook without background pdfs

1f257b2

Support templates

c29be12

Updated colors and docu

94d1be9

Deleted unused code

b805445

Removed logging statements

f3d4542

Added mgeta to get a folder annotated

a57cbaa

Updated readme for mgeta

ba6d0a5

Minor changes

5662904

Test with different svg to pdf conversion

3b978a1

Test with inkscape

ac4b5c0

Store template in template tmp folder

a45c4fd

Using rsvg-convert as its faster.

982d8f2

Update README.md

cb7771f

peerdavid added 4 commits November 7, 2018 13:45

Update README.md

cbc6c0c

Update README.md

b09390d

Prepare for PR

2315caa

Merge remote-tracking branch 'origin/master'

c6b6aa0

peerdavid mentioned this pull request Dec 4, 2018

get pdf with annotations #30

Closed

peerdavid added 3 commits December 23, 2018 11:36

Update annotated pdf file only if modified date time changed.

0dfee84

Better logging

43bc787

Improved performance

f872dfd

juruen suggested changes Dec 23, 2018

View reviewed changes

peerdavid added 4 commits December 23, 2018 12:41

Fixed invalid logging

0c857d9

Ignore epub files and simply move pdfs that are not annotated.

ea56392

Removed dependency to pdfunite

e434018

1. Added verification for geta. If tools are missing geta and mgeta a…

9fbfb30

…re disabled 2. Updated README.md to be more explicit about the optional commands geta and mgeta

peerdavid changed the title ~~Download annotated notebooks~~ [WIP] Download annotated notebooks Dec 24, 2018

Rotate annotations whenever the pdf file is rotated

b36b097

juruen reviewed Jan 1, 2019

View reviewed changes

README.md Outdated

# Thanks to

[1] rmapi, https://github.com/juruen/rmapi <br />

Copy link

Owner

juruen Jan 1, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can drop this line :)

juruen mentioned this pull request Jan 4, 2019

Initial support for annotations #38

Merged

Minor fix

964ed36

uising github.com/peerdavid; fixed 1.7 bug with missing page count

3ca7ec1

peerdavid closed this Jul 20, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[WIP] Download annotated notebooks #34

[WIP] Download annotated notebooks #34

peerdavid commented Dec 4, 2018 •

edited

Loading

juruen commented Dec 8, 2018

peerdavid commented Dec 8, 2018

juruen left a comment

juruen Dec 23, 2018

juruen Dec 23, 2018

juruen Dec 23, 2018

peerdavid Dec 23, 2018

juruen Dec 23, 2018

peerdavid Dec 23, 2018

juruen Jan 1, 2019

mseri commented Apr 11, 2019

peerdavid commented Apr 11, 2019

lobre commented Apr 11, 2019 •

edited

Loading

mseri commented Apr 11, 2019 •

edited

Loading

martin-braun commented Jan 28, 2021

peerdavid commented Jan 28, 2021



		# Thanks to
		[1] rmapi, https://github.com/juruen/rmapi <br />

[WIP] Download annotated notebooks #34

[WIP] Download annotated notebooks #34

Conversation

peerdavid commented Dec 4, 2018 • edited Loading

juruen commented Dec 8, 2018

peerdavid commented Dec 8, 2018

juruen left a comment

Choose a reason for hiding this comment

juruen Dec 23, 2018

Choose a reason for hiding this comment

juruen Dec 23, 2018

Choose a reason for hiding this comment

juruen Dec 23, 2018

Choose a reason for hiding this comment

peerdavid Dec 23, 2018

Choose a reason for hiding this comment

juruen Dec 23, 2018

Choose a reason for hiding this comment

peerdavid Dec 23, 2018

Choose a reason for hiding this comment

juruen Jan 1, 2019

Choose a reason for hiding this comment

mseri commented Apr 11, 2019

peerdavid commented Apr 11, 2019

lobre commented Apr 11, 2019 • edited Loading

mseri commented Apr 11, 2019 • edited Loading

martin-braun commented Jan 28, 2021

peerdavid commented Jan 28, 2021

peerdavid commented Dec 4, 2018 •

edited

Loading

lobre commented Apr 11, 2019 •

edited

Loading

mseri commented Apr 11, 2019 •

edited

Loading