Use unquote to convert spaces. Fixes #151.

[ccsplit]

Since the information is passed back and forth through JSON with the agent
it is URL-Encoded and therefore it needs to be removed when passing the
string to an local_operation/remote_operation path command.

Fixed

Bug fixes proposed in this pull request:

• Errors running modules targeting an application with a space character in the name #151 by unquoting the paths when they are retrieved from the plist on the needle-agent.

[marco-lancini]

Hi @ccsplit, this fix has already been proposed in #155: although this proposed fix works for apps containing a space in their name, this doesn't work if the app folder contains other special characters (e.g., accents)

[ccsplit]

Ah, I had not seen that PR since it didn't reference the issue. @marco-lancini, is there an example of the other special characters available publicly within an application? If not I will work on creating a test ipa for the issue and amend it to my proposed fix.

[marco-lancini]

thank you @ccsplit, that would help a lot!

[ccsplit]

@marco-lancini, I believe the accent/backquote ` issue might be a little bit more involved as it looks like the shell, local or remote is attempting to interpret the character and execute code found within it or failing when there is only one. Similar to when you do something like: vim `which python` , etc. I've attempted to add an escape for these to Utils.escape_path but it looks like it requires escape in some instances but not others. For example within the module binary/info/metadata it requires one when calling __parse_plist_info because the path is ran through Utils.escape_path twice, here and here. However, for the next call within __detect_architecture it is only escaped once within the __detect_architecture method and fails if the accent/backquote is escaped.

I've been using ios-testapp-specialchars when testing this issue if you want to take a look. Here is a gist of the current changes I've been working around with, I've also created a branch issue/#151-specialcharacters within my fork if you'd rather just pull it down. If I figure out a better way I'll update this PR unless you want to create a separate issue for this?

[marco-lancini]

Hi @ccsplit, thanks for your effort! Could you just commit everything to this PR, so I can review it in one go?

[ccsplit]

An attempt at trying to resolve issues with accents/backquotes. Currently this does not work, and will actually cause #161 to occur more frequently during the re-attempts. Also once it gets into that state it will error when trying to disconnect and clean up the device causing needle to hang. In which you will need to use kill <needlePID> to stop it.

[marco-lancini]

what about accented characters like: É?

[ccsplit]

@marco-lancini, I've added the changes I had when working through the accent/backquote issue to the PR.

[marco-lancini]

Hi @ccsplit, I tried your proposed fix, but when I try to analyze an app which has the following character in its name (è) I get the following error:

[*] Retrieving app's metadata...
------------------------------------------------------------
Traceback (most recent call last):
  File "/root/needle/core/framework/module.py", line 111, in do_run
    pre = self.module_pre()
  File "/root/needle/core/framework/module.py", line 147, in module_pre
    if self.app_check() is None: return None
  File "/root/needle/core/framework/framework.py", line 690, in app_check
    self.APP_METADATA = Framework.APP_METADATA = self.device.app.get_metadata(app)
  File "/root/needle/core/device/app.py", line 19, in get_metadata
    return self._retrieve_metadata()
  File "/rootneedle/core/device/app.py", line 27, in _retrieve_metadata
    plist_info_path = Utils.escape_path('%s/Info.plist' % metadata_agent['binary_directory'], escape_accent=True)
  File "/root/needle/core/utils/utils.py", line 29, in escape_path
    return pipes.quote(path)
UnicodeEncodeError: 'ascii' codec can't encode characters in position 89-90: ordinal not in range(128)
------------------------------------------------------------
[!] UnicodeEncodeError: 'ascii' codec can't encode characters in position 89-90: ordinal not in range(128)

[ccsplit]

@marco-lancini, does this occur prior to my change as well? If so you would need to use 'string'.decode('utf-8') or one of the other encodings (I'd suggest just using UTF-8 though or whatever iOS uses). If it is only after my change that is weird since I don't see anything within my change that should affect that. Unless re.sub returns the text in ascii whereas it was normally unicode.

[marco-lancini]

I've been banging my head on this for quite a while now. Even encodings doesn't seem to solve the issue. Any help would be massively appreciated!

[ccsplit]

I'll take a look once and see what I can figure out.

[ccsplit]

@marco-lancini, It looks like this is similar to #80 and/or #118. With DEBUG == True it appears to fail when trying to print the values and tries to use the default codec ascii to format the characters. I was able to resolve this by changing "some text {}".format(value) to u"some text {}".format(value). However, needle then had issues with finding ~/.needle/tmp/plist, because the scp failed. This was using the ios-testapp-specialchars test app I created/changed to include some other characters.

[flamecopper]

Can I confirm that I just need to change Example App.app to 'Example App.app'.decode('utf-8').
I am sorry, I am not sure of exact solution .