Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Change url companies param on :show route to name attribute #594

Closed

Conversation

DouglasLutz
Copy link

Fixes #588

@gemantzu @doomspork I'm coming from rails and don't have experience with elixir/phoenix so if there is a better way to do this I'm open for ideas :D

@DouglasLutz DouglasLutz requested a review from a team as a code owner March 4, 2020 16:25
Copy link
Collaborator

@gemantzu gemantzu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @DouglasLutz, thanks for the PR. I am a little worried about how that looks with companies with multiple word names, and not from the context perspective, but if it will actually work with the name being used as a param. How would the multiple word names look like as a url? Let me check this in the afternoon and come back at you.
@doomspork thoughts?

@maartenvanvliet
Copy link
Member

Agreed with @gemantzu, using proper slugs would be preferable. There's no uniqueness enforced on company name either, so if we have duplicates they would result in unreachable urls.

@doomspork
Copy link
Member

@DouglasLutz I'm glad you've decided to jump into Elixir head first and get right to contributing 🎉

I think @gemantzu and @maartenvanvliet have good points. Would you like some help making these changes or do you want to give it a shot and we can go from there?

Thanks again for getting involved, we're stoked to have you 😀

@DouglasLutz
Copy link
Author

Thanks for the feedback, I made some research and think I can create a slug for the companies

@DouglasLutz DouglasLutz force-pushed the change-company-url-param branch 2 times, most recently from 1667cde to c6bb282 Compare March 5, 2020 21:16
@@ -0,0 +1,23 @@
defmodule Companies.Repo.Migrations.AddSlugValueForExistingCompanies do
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know how Heroku works with elixir, but when I tried to do this in the past, while it worked for my local environment, it failed miserably for production, since the migration scripts did not have any access to the actual app modules when they run, and therefore they failed. @doomspork / @maartenvanvliet ?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, locally it worked but I don't know about production env. An alternative to this would be run this slug generation one time in heroku.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, restricting the migrations to just ddl might be preferable. There was an interesting post by AppSignal on doing this in Elixir (https://blog.appsignal.com/2020/02/25/migrating-production-data-in-elixir.html)

That might be overkill in this case and a one time command would also work.

company
|> cast(attrs, [:name, :description, :url, :github, :location, :blog, :industry_id])
|> cast(attrs, [:name, :description, :url, :github, :location, :blog, :industry_id, :slug])
|> validate_required([:name, :description, :url, :industry_id])
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The slug is unique in the database by a constraint but this constraint is not handled in the changeset.

Another thing to deal with is how pending changes are managed in the code. When a company is created a pending_change is inserted in the database. When that change is approved, the company will be inserted in the companies table and could fail on the uniqueness constraint.

It would be preferable imo to deal with duplicate slugs earlier, perhaps by adding an unsafe_validate_unique and adding postfixes to the slugs to make them unique.

This would not entirely be foolproof (e.g. multiple pending changes could have the same slug) but could go a long way.

Copy link
Collaborator

@gemantzu gemantzu Mar 7, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we could add the company id in the slug just to be safe. e.g. Plataformatec turns into 1-plataformatec (the id here is random).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Crazy idea feel free to shoot this down: what if we don't enforce the uniqueness in the changeset (or we create a changeset without it) and instead defer the work of creating unique slugs onto admins approving changes. In the UI to approve a company we could show whether the slug was already taken and require the admin to make a unique one.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@maartenvanvliet / @gemantzu how do you both feel about moving generation of things like slugs (or password hashes) into the changeset pipeline like such?

def changeset(company, attrs) do
  company
  |> cast(attrs, [:name, :description, :url, :github, :location, :blog, :industry_id, :slug])
  |> validate_required([:name, :description, :url, :industry_id])
  |> generate_slug()
  |> unique_constraint(:slug)
end

# when there is an existing non-nil `id` in the data, we don't want to regenerate a slug even on name change
# that would break any potential backlinks.
defp generate_slug(%Ecto.Changeset{data: %{id: id}} = changeset) when not is_nil(id) do
  changeset
end

defp generate_slug(%Ecto.Changeset{changes: %{name: name}} = changeset) do
  slug =
    name
    |> String.replace(~r/['’]s/u, "s")
    |> String.downcase()
    |> String.replace(~r/([^a-z0-9가-힣])+/, "-")
    |> String.replace(" ", "-")

  put_change(changeset, :slug, slug)
end

resources "/jobs", JobController, except: [:index, :show]
resources "/users", UserController, only: [:edit, :update]
end

get "/", CompanyController, :recent
resources "/companies", CompanyController, only: [:index]
resources "/companies", CompanyController, only: [:show], param: "name"
resources "/companies", CompanyController, only: [:index, :show], param: "slug"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just to be aware of this, the old url's based on integers would no longer work with this change. We could implement a fallback but this is also fine.

@@ -6,13 +6,15 @@ defmodule Companies.Schema.Company do

alias Companies.Schema.{Industry, Job, PendingChange}

@derive {Phoenix.Param, key: :slug}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like this :)

@gemantzu
Copy link
Collaborator

@DouglasLutz need any help with this? Can we provide support somehow if you are stuck?

@DouglasLutz
Copy link
Author

DouglasLutz commented Mar 12, 2020

@DouglasLutz need any help with this? Can we provide support somehow if you are stuck?

@gemantzu
Sorry, have been a little busy this week but yes, I got stuck in the assurance of generating unique slugs. I thought about appending a number in the slug if the slug is already in use, but this would add queries to company model and I don't know if it's a good thing to do so I think I'll take your suggestion of adding the company id to the slug.

@gemantzu
Copy link
Collaborator

@DouglasLutz no worries, I know how it is with work and life sometimes. I just thought I'd check in to see if there was anything I could help with. There's no rush.

If you want to push up a partial solution I'd be happy to take a look. We can bounce ideas off each other if need be but I'm sure whatever you decide on is fine.

We don't need to be too worried about performance. We can always come back and make improvements.

** (Ecto.NoResultsError)

"""
def get_by_slug!(slug, opts \\ []) do
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@DouglasLutz / @maartenvanvliet what do you think about doing away with this function and instead amending get!/1 like such:

def get!(key, opts \\ []) do
  preloads = Keyword.get(opts, :preloads, [])

  query =
    from(c in Company)
    |> preload(^preloads)
    |> from()
    |> where([c], is_nil(c.removed_pending_change_id))

  final_query =
    case Integer.parse(key) do
      :error -> where(query, [c], c.slug == ^key)
      {int_id, _remainder} -> where([c], c.id == ^int_id or c.slug == ^key) 
    end

  Repo.one!(final_query)
end

While this function is a bit more complicated, I'm wondering if simplify things. We no longer need to change the param key to "slug" and update all occurrences of "id" accordingly. We handle both slug and id searches. All we need is the migration, schema + changeset, and this. 🤔

What do you both think?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I had a local proof of concept with slugs that worked more along these lines. I see you account for slugs being numbers, I didn't :P

Advantage is old integer based urls continue to work and a simpler internal api.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see you account for slugs being numbers, I didn't :P

It hit me last night that there could realistically be a company who's name is just digits and that could throw a wrench in things.

While this code works I can't help but feel this is still "wrong". I think we should be trying to identify when someone uses the old path (w/ ID) and 301 them to the new slug path. Thoughts?

@@ -24,8 +26,23 @@ defmodule Companies.Schema.Company do

@doc false
def changeset(company, attrs) do
attrs = Map.merge(attrs, slug_map(attrs))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we flip these around we can avoid nesting in favor of a pipeline 🎉

attrs
|> slug_map()
|> Map.merge(attrs)

company
|> cast(attrs, [:name, :description, :url, :github, :location, :blog, :industry_id])
|> cast(attrs, [:name, :description, :url, :github, :location, :blog, :industry_id, :slug])
|> validate_required([:name, :description, :url, :industry_id])
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@maartenvanvliet / @gemantzu how do you both feel about moving generation of things like slugs (or password hashes) into the changeset pipeline like such?

def changeset(company, attrs) do
  company
  |> cast(attrs, [:name, :description, :url, :github, :location, :blog, :industry_id, :slug])
  |> validate_required([:name, :description, :url, :industry_id])
  |> generate_slug()
  |> unique_constraint(:slug)
end

# when there is an existing non-nil `id` in the data, we don't want to regenerate a slug even on name change
# that would break any potential backlinks.
defp generate_slug(%Ecto.Changeset{data: %{id: id}} = changeset) when not is_nil(id) do
  changeset
end

defp generate_slug(%Ecto.Changeset{changes: %{name: name}} = changeset) do
  slug =
    name
    |> String.replace(~r/['’]s/u, "s")
    |> String.downcase()
    |> String.replace(~r/([^a-z0-9가-힣])+/, "-")
    |> String.replace(" ", "-")

  put_change(changeset, :slug, slug)
end

add :slug, :string
end

create index(:companies, [:slug], unique: true)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's change this to use the unique_index/2: create unique_index(:companies, [:slug])

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was contemplating this as well. So 👍

@gemantzu
Copy link
Collaborator

@doomspork a good solution to me is provided by the phoenix book.

  1. we add the slug field, non-unique
  2. we add the field in the ecto schema
  3. we add a new pipe in the changeset pipeline for companies, where we produce the slug based on the company name (replace spaces with -, downcase, etc)
  4. we create a new Phoenix.Param implementation for Company like so:
defimpl Phoenix.Param, for: Companies.Schema.Company do
  def to_param(%{slug: slug, id: id}) do
    "#{id}-#{slug}"
  end
end

Then, when we call Routes.company_path, the url provided is in the form of /companies/1-plataformatec
5. we create a new behaviour implementation for Ecto.Type for our ids like so:

defmodule Companies.Permalink do
  @behaviour Ecto.Type
  
  def type, do: :id
  
  def cast(binary) when is_binary(binary) do
     case Integer.parse(binary) do
       {int, _} when int > 0 -> {:ok, int}
       _ -> :error
     end
  end
  
  def cast(integer) when is_integer(integer) do
    {:ok, integer}
  end
  
  def cast(_) do
    :error
  end

  def dump(integer) when is_integer(integer) do
    {:ok, integer}
  end
  
  def load(integer) when is_integer(integer) do
    {:ok, integer}
  end
end
  1. then we finally modify the default way our primary keys work:
@primary_key {:id, Rumbl.Multimedia.Permalink, autogenerate: true}
schema "companies" do
...

The end result is that we don't have to do anything about the context methods, they work as intended.

@doomspork
Copy link
Member

@gemantzu if we default to putting a number on the slug then moving to slugs makes little sense to me. People are attempting to file companies by their exact name, they will never be able to guess the internal primary key for a company. Right or am I confused?

@DouglasLutz
Copy link
Author

Also if in this case the purpose of adding slugs is to people put the name of the company in the url, a simpler solution could be by doing something like this in the controller. And also this would not break the existing routes.

def show(conn, %{"id" => id}) do
    case Integer.parse(id) do
      :error ->
        redirect(conn, to: "/en/companies?search[text]=#{id}")
      {_num, _remainder} ->
        company = Companies.get!(id, preloads: [:jobs, :industry])
        render(conn, "show.html", company: company)
    end
  end

Of course this would not solve the cases of someone trying to go to /companies/plataformatec/edit but I feel that if people are already in a company#show page and want to edit it they would just add the /edit in the end of url.

@doomspork @gemantzu @maartenvanvliet What are your thoughts on this?

@doomspork
Copy link
Member

@DouglasLutz your proposed got me thinking more about moving this into the controller. We want to support slugs and ids in URL so we should see if we can't figure that out before moving this to a search redirect.

One issue we have is figure out which type we have and passing it down to the query. What if we change get!/3 (which uses Repo.get!/3) to both use and be called get_by!/3, something this like:

@doc """
iex> get_by!(id: 123)
%Company{}

iex> get_by!(id: 456)
** (Ecto.NoResultsError)

iex> get_by!(slug: "example-company")
%Company{}

"""
def get_by!(predicates, opts \\ []) do
  preloads = Keyword.get(opts, :preloads, [])

  from(c in Company)
  |> preload(^preloads)
  |> from()
  |> where([c], is_nil(c.removed_pending_change_id))
  |> Repo.get_by!(predicates)
end

In our controller could do something like this:

def show(conn, %{"id" => id_or_slug}) do
  case Integer.parse(id_or_slug) do
    {int_id, _remainder} ->
      %{slug: slug} = Companies.get!(id: int_id)
      redirect(conn, to: Routes.company_path(conn, :show, slug))

    :error ->
      company = Companies.get!(slug: id_or_slug)
      render(conn, "show.html", company: company)
  end
end

What do you think? Since we will get more slugs going forward than id type URLs we'll need to be considerate of the Integer.parse/1 performance.

@DouglasLutz DouglasLutz changed the title Change url companies param on :show route to name attribute [WIP] Change url companies param on :show route to name attribute Mar 18, 2020
@gemantzu
Copy link
Collaborator

gemantzu commented Apr 18, 2020

@DouglasLutz thanks again for working on this. Are you done? Should we check again the code and functionality?

@doomspork
Copy link
Member

@DouglasLutz this is nearly 3 years old and there's quite a few conflicts. I'm going to close but I think we should look to see if this change is still applicable and if so, open a new PR with the feedback incorporated 🎉

@doomspork doomspork closed this Mar 2, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

company slugs
4 participants