Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(server, builder): OAuth support + API key management + GitHub blocks #8044

Open
wants to merge 26 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 12 commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
efdd71b
Add Github blocks + "DEVELOPER_TOOLS" category
Bentlybro Sep 10, 2024
57c22e0
Merge branch 'master' into reinier/open-1806-implement-front-to-back-…
Pwuts Sep 11, 2024
4cec829
move github_blocks.py -> github.py
Pwuts Sep 11, 2024
e7fa682
remove duplicate OAuth handler files
Pwuts Sep 11, 2024
8ae5378
feat(server): Add Supabase support to `AppService`
Pwuts Sep 11, 2024
20c5137
Add `**kwargs` to `Block.run(..)` signature to support additional kwargs
Pwuts Sep 11, 2024
08666f4
feat(libs/supabase_integration_credentials_store): Add `CredentialsTy…
Pwuts Sep 11, 2024
14c8ca4
Add strict support for `credentials` fields on blocks
Pwuts Sep 11, 2024
da6d443
feat(executor): Fetch credentials for graph execution and pass them d…
Pwuts Sep 12, 2024
56d379f
feat(server/blocks): Integrate GitHub blocks with new credentials inf…
Pwuts Sep 12, 2024
d7e34b1
Merge branch 'master' into reinier/open-1806-implement-front-to-back-…
Pwuts Sep 12, 2024
c623d61
Merge branch 'master' into reinier/open-1806-implement-front-to-back-…
Pwuts Sep 12, 2024
26a837a
Merge branch 'master' into reinier/open-1806-implement-front-to-back-…
Pwuts Sep 16, 2024
c8f0110
Fix schema output with extra properties on complex input fields
Pwuts Sep 16, 2024
4a56de0
Merge branch 'master' into reinier/open-1806-implement-front-to-back-…
Pwuts Sep 17, 2024
57d543a
feat(server): Server endpoints to add `APIKeyCredentials` and remove …
kcze Sep 18, 2024
03fe5a1
Merge branch 'master' into reinier/open-1806-implement-front-to-back-…
Pwuts Sep 18, 2024
f4a92dc
fix conflict resolve in integrations API router
Pwuts Sep 18, 2024
c8115b1
Merge branch 'master' into reinier/open-1806-implement-front-to-back-…
Pwuts Sep 18, 2024
1b54bd3
feat(builder): Authorization UI on for integration blocks (#8067)
kcze Sep 19, 2024
d9b2ccc
fix field description on API key dialog
Pwuts Sep 19, 2024
e260c39
parametrize `test_available_blocks`
Pwuts Sep 19, 2024
a5264d6
Update .env.example and docker-compose.yml with new config variables
Pwuts Sep 19, 2024
74138d7
add docs for adding authenticated blocks
Pwuts Sep 19, 2024
57afc85
add comment to .env.example about GitHub app type
Pwuts Sep 19, 2024
6aea79b
fix block tests with credentials
Pwuts Sep 19, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/content/server/new_blocks.md
Original file line number Diff line number Diff line change
Expand Up @@ -84,7 +84,7 @@ Follow these steps to create and test a new block:
5. **Implement the `run` method with error handling:**, this should contain the main logic of the block:

Copy link
Contributor

@aarushik93 aarushik93 Sep 19, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the GitHub integration to work, the back end now needs GITHUB_CLIENT_ID and GITHUB_CLIENT_SECRET

In the docs and readme, can we add a section for "integration blocks" or wtv we are calling them and start adding this kind of information about configuring them?

Copy link
Member Author

@Pwuts Pwuts Sep 19, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done: 74138d7

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't update the README because:

  • I want to avoid duplicating documentation
  • The extra stuff for credentials technically fits within the summary instructions for adding blocks that's currently in the README

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks - what about also adding somewhere - where I go to find client id and secret and API key?
I was thinking we'd just have a subsection somewhere in docs for each integration that just tells you where to go

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cause rn I can't find exactly where to go for the gh client id and secret....

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added to .env.example: 57afc85

```python
def run(self, input_data: Input) -> BlockOutput:
def run(self, input_data: Input, **kwargs) -> BlockOutput:
try:
topic = input_data.topic
url = f"https://en.wikipedia.org/api/rest_v1/page/summary/{topic}"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -27,20 +27,29 @@ class OAuth2Credentials(_BaseCredentials):
scopes: list[str]
metadata: dict[str, Any] = Field(default_factory=dict)

def bearer(self) -> str:
return f"Bearer {self.access_token.get_secret_value()}"


class APIKeyCredentials(_BaseCredentials):
type: Literal["api_key"] = "api_key"
api_key: SecretStr
expires_at: Optional[int]
"""Unix timestamp (seconds) indicating when the API key expires (if at all)"""

def bearer(self) -> str:
return f"Bearer {self.api_key.get_secret_value()}"


Credentials = Annotated[
OAuth2Credentials | APIKeyCredentials,
Field(discriminator="type"),
]


CredentialsType = Literal["api_key", "oauth2"]


class OAuthState(BaseModel):
token: str
provider: str
Expand Down
17 changes: 8 additions & 9 deletions rnd/autogpt_server/autogpt_server/blocks/basic.py
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ def __init__(self):
static_output=True,
)

def run(self, input_data: Input) -> BlockOutput:
def run(self, input_data: Input, **kwargs) -> BlockOutput:
Pwuts marked this conversation as resolved.
Show resolved Hide resolved
yield "output", input_data.data or input_data.input


Expand All @@ -79,13 +79,12 @@ def __init__(self):
test_output=("status", "printed"),
)

def run(self, input_data: Input) -> BlockOutput:
def run(self, input_data: Input, **kwargs) -> BlockOutput:
print(">>>>> Print: ", input_data.text)
yield "status", "printed"


class FindInDictionaryBlock(Block):

class Input(BlockSchema):
input: Any = Field(description="Dictionary to lookup from")
key: str | int = Field(description="Key to lookup in the dictionary")
Expand Down Expand Up @@ -119,7 +118,7 @@ def __init__(self):
categories={BlockCategory.BASIC},
)

def run(self, input_data: Input) -> BlockOutput:
def run(self, input_data: Input, **kwargs) -> BlockOutput:
obj = input_data.input
key = input_data.key

Expand Down Expand Up @@ -201,7 +200,7 @@ def __init__(self):
ui_type=BlockUIType.INPUT,
)

def run(self, input_data: Input) -> BlockOutput:
def run(self, input_data: Input, **kwargs) -> BlockOutput:
yield "result", input_data.value


Expand Down Expand Up @@ -284,7 +283,7 @@ def __init__(self):
ui_type=BlockUIType.OUTPUT,
)

def run(self, input_data: Input) -> BlockOutput:
def run(self, input_data: Input, **kwargs) -> BlockOutput:
"""
Attempts to format the recorded_value using the fmt_string if provided.
If formatting fails or no fmt_string is given, returns the original recorded_value.
Expand Down Expand Up @@ -344,7 +343,7 @@ def __init__(self):
],
)

def run(self, input_data: Input) -> BlockOutput:
def run(self, input_data: Input, **kwargs) -> BlockOutput:
try:
# If no dictionary is provided, create a new one
if input_data.dictionary is None:
Expand Down Expand Up @@ -415,7 +414,7 @@ def __init__(self):
],
)

def run(self, input_data: Input) -> BlockOutput:
def run(self, input_data: Input, **kwargs) -> BlockOutput:
try:
# If no list is provided, create a new one
if input_data.list is None:
Expand Down Expand Up @@ -456,5 +455,5 @@ def __init__(self):
ui_type=BlockUIType.NOTE,
)

def run(self, input_data: Input) -> BlockOutput:
def run(self, input_data: Input, **kwargs) -> BlockOutput:
yield "output", input_data.text
2 changes: 1 addition & 1 deletion rnd/autogpt_server/autogpt_server/blocks/block.py
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ def __init__(self):
disabled=True,
)

def run(self, input_data: Input) -> BlockOutput:
def run(self, input_data: Input, **kwargs) -> BlockOutput:
code = input_data.code

if search := re.search(r"class (\w+)\(Block\):", code):
Expand Down
2 changes: 1 addition & 1 deletion rnd/autogpt_server/autogpt_server/blocks/branching.py
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,7 @@ def __init__(self):
],
)

def run(self, input_data: Input) -> BlockOutput:
def run(self, input_data: Input, **kwargs) -> BlockOutput:
value1 = input_data.value1
operator = input_data.operator
value2 = input_data.value2
Expand Down
2 changes: 1 addition & 1 deletion rnd/autogpt_server/autogpt_server/blocks/csv.py
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ def __init__(self):
],
)

def run(self, input_data: Input) -> BlockOutput:
def run(self, input_data: Input, **kwargs) -> BlockOutput:
import csv
from io import StringIO

Expand Down
6 changes: 3 additions & 3 deletions rnd/autogpt_server/autogpt_server/blocks/discord.py
Original file line number Diff line number Diff line change
Expand Up @@ -81,14 +81,14 @@ async def on_message(message):

await client.start(token)

def run(self, input_data: "ReadDiscordMessagesBlock.Input") -> BlockOutput:
def run(self, input_data: Input, **kwargs) -> BlockOutput:
while True:
for output_name, output_value in self.__run(input_data):
yield output_name, output_value
if not input_data.continuous_read:
break

def __run(self, input_data: "ReadDiscordMessagesBlock.Input") -> BlockOutput:
def __run(self, input_data: Input) -> BlockOutput:
try:
loop = asyncio.get_event_loop()
future = self.run_bot(input_data.discord_bot_token.get_secret_value())
Expand Down Expand Up @@ -187,7 +187,7 @@ def chunk_message(self, message: str, limit: int = 2000) -> list:
"""Splits a message into chunks not exceeding the Discord limit."""
return [message[i : i + limit] for i in range(0, len(message), limit)]

def run(self, input_data: "SendDiscordMessageBlock.Input") -> BlockOutput:
def run(self, input_data: Input, **kwargs) -> BlockOutput:
try:
loop = asyncio.get_event_loop()
future = self.send_message(
Expand Down
2 changes: 1 addition & 1 deletion rnd/autogpt_server/autogpt_server/blocks/email_block.py
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,7 @@ def send_email(
except Exception as e:
return f"Failed to send email: {str(e)}"

def run(self, input_data: Input) -> BlockOutput:
def run(self, input_data: Input, **kwargs) -> BlockOutput:
status = self.send_email(
input_data.creds,
input_data.to_email,
Expand Down
Loading
Loading