[feature] overhaul the oidc system (#961)

* [feature] overhaul the oidc system

this allows for more flexible username handling and prevents account
takeover using old email addresses

* [feature] add migration path for old OIDC users

* [feature] nicer error reporting for users

* [docs] document the new OIDC flow

* [fix] return early on oidc error

* [docs]: add comments on the finalization logic

cmd/gotosocial/action/admin/account/account.go

@@ -78,7 +78,7 @@
 		return err
 	}
 
-	_, err = dbConn.NewSignup(ctx, username, "", false, email, password, nil, "", "", true, false)
+	_, err = dbConn.NewSignup(ctx, username, "", false, email, password, nil, "", "", true, "", false)
 	if err != nil {
 		return err
 	}

docs/configuration/oidc.md

@@ -68,6 +68,13 @@ oidc-scopes:
   - "email"
   - "profile"
   - "groups"
+
+# Bool. Link OIDC authenticated users to existing ones based on their email address.
+# This is mostly intended for migration purposes if you were running previous versions of GTS
+# which only correlated users with their email address. Should be set to false for most usecases.
+# Options: [true, false]
+# Default: false
+oidc-link-existing: false
 ```
 
 ## Behavior
@@ -76,41 +83,19 @@ When OIDC is enabled on GoToSocial, the default sign-in page redirects automatic
 
 This means that OIDC essentially *replaces* the normal GtS email/password sign-in flow.
 
-When a user logs in through OIDC, GoToSocial will request that user's preferred email address and username from the OIDC provider. It will then use the returned email address to either:
+Due to the way the ActivityPub standard works, you _cannot_ change your username
+after it has been set. This conflicts with the OIDC spec which does not
+guarantee that the `preferred_username` field is stable.
 
-*If the email address is already associated with a user/account*: sign the requester in as that user/account.
+To work with this, we ask the user to provide a username on their first login
+attempt. The field for this is pre-filled with the value of the `preferred_username` claim.
 
-Or:
+After authenticating, GtS stores the `sub` claim supplied by the OIDC provider.
+On subsequent authentication attempts, the user is looked up using this claim
+exclusively.
 
-*If the email address is not yet associated with a user/account*: create a new user and account with the returned credentials, and sign the requester in as that user/account.
+This then allows you to change the username on a provider level without losing
-
+access to your GtS account.
-In other words, GoToSocial completely delegates sign-in authority to the OIDC provider, and trusts whatever credentials it returns.
-
-### Username conflicts
-
-In some cases, such as when a server has been switched to use OIDC after already using default settings for a while, there may be an overlap between usernames returned from OIDC, and usernames that already existed in the database.
-
-For example, let's say that someone with username `gordonbrownfan` and email address `gordon_is_best@example.org` has an account on a GtS instance that uses the default sign-in flow.
-
-That GtS instance then switches to using OIDC login. However, in the OIDC's storage there's also a user with username `gordonbrownfan`. If this user has the email address `gordon_is_best@example.org`, then GoToSocial will assume that the two users are the same and just log `gordonbrownfan` in as though nothing had changed. No problem!
-
-However, if the user in the OIDC storage has a different email address, GoToSocial will try to create a new user and account for this person.
-
-Since the username `gordonbrownfan` is already taken, GoToSocial will try `gordonbrownfan1`. If this is also taken, it will try `gordonbrownfan2`, and so on, until it finds a username that's not yet taken. It will then sign the requester in as that user/account, distinct from the original `gordonbrownfan`.
-
-### Malformed usernames
-
-A username returned from an OIDC provider might not always fit the pattern of what GoToSocial accepts as a valid username, ie., lower-case letters, numbers, and underscores. In this case, GoToSocial will do its best to parse the returned username into something that fits the pattern.
-
-For example, say that an OIDC provider returns the username `Marx Is Great` for a sign in, which doesn't fit the pattern because it contains upper-case letters and spaces.
-
-In this case, GtS will convert it into `marx_is_great` by applying the following rules:
-
-1. Trim any leading or trailing whitespace.
-2. Convert all letters to lowercase.
-3. Replace spaces with underscores.
-
-Unfortunately, at this point GoToSocial doesn't know how to handle returned usernames containing special characters such as `@` or `%`, so these will return an error.
 
 ### Group membership
 
@@ -118,6 +103,16 @@ Most OIDC providers allow for the concept of groups and group memberships in ret
 
 If the returned OIDC groups information for a user contains membership of the groups `admin` or `admins`, then that user will be created/signed in as though they are an admin.
 
+## Migrating from old versions
+
+If you're moving from an old version of GtS which used the unstable `email`
+claim for unique user identification, you can set the `oidc-link-existing`
+configuration to `true`. If no user can be found for the ID returned by the
+provider, a lookup based on the `email` claim is performed instead. If this
+succeeds, the stable id is added to the database for the matching user.
+
+You should only use this for a limited time to avoid malicious account takeover.
+
 ## Provider Examples
 
 ### Dex

example/config.yaml

@@ -490,6 +490,13 @@ oidc-scopes:
   - "profile"
   - "groups"
 
+# Bool. Link OIDC authenticated users to existing ones based on their email address.
+# This is mostly intended for migration purposes if you were running previous versions of GTS
+# which only correlated users with their email address. Should be set to false for most usecases.
+# Options: [true, false]
+# Default: false
+oidc-link-existing: false
+
 #######################
 ##### SMTP CONFIG #####
 #######################

internal/api/client/auth/auth.go

@@ -50,6 +50,9 @@
 	// OauthAuthorizePath is the API path for authorization requests (eg., authorize this app to act on my behalf as a user)
 	OauthAuthorizePath = "/oauth/authorize"
 
+	// OauthFinalizePath is the API path for completing user registration with additional user details
+	OauthFinalizePath = "/oauth/finalize"
+
 	// CallbackPath is the API path for receiving callback tokens from external OIDC providers
 	CallbackPath = oidc.CallbackPath
 
@@ -64,6 +67,8 @@
 	sessionScope         = "scope"
 	sessionInternalState = "internal_state"
 	sessionClientState   = "client_state"
+	sessionClaims        = "claims"
+	sessionAppID         = "app_id"
 )
 
 // Module implements the ClientAPIModule interface for
@@ -93,6 +98,7 @@ func (m *Module) Route(s router.Router) error {
 	s.AttachHandler(http.MethodPost, OauthAuthorizePath, m.AuthorizePOSTHandler)
 
 	s.AttachHandler(http.MethodGet, CallbackPath, m.CallbackGETHandler)
+	s.AttachHandler(http.MethodPost, OauthFinalizePath, m.FinalizePOSTHandler)
 
 	s.AttachHandler(http.MethodGet, oauth.OOBTokenPath, m.OobHandler)
 	return nil

internal/api/client/auth/callback.go

@@ -24,13 +24,13 @@
 	"fmt"
 	"net"
 	"net/http"
-	"strconv"
 	"strings"
 
 	"github.com/gin-contrib/sessions"
 	"github.com/gin-gonic/gin"
 	"github.com/google/uuid"
 	"github.com/superseriousbusiness/gotosocial/internal/api"
+	"github.com/superseriousbusiness/gotosocial/internal/config"
 	"github.com/superseriousbusiness/gotosocial/internal/db"
 	"github.com/superseriousbusiness/gotosocial/internal/gtserror"
 	"github.com/superseriousbusiness/gotosocial/internal/gtsmodel"
@@ -39,6 +39,12 @@
 	"github.com/superseriousbusiness/gotosocial/internal/validate"
 )
 
+// extraInfo wraps a form-submitted username and transmitted name
+type extraInfo struct {
+	Username string `form:"username"`
+	Name     string `form:"name"` // note that this is only used for re-rendering the page in case of an error
+}
+
 // CallbackGETHandler parses a token from an external auth provider.
 func (m *Module) CallbackGETHandler(c *gin.Context) {
 	s := sessions.Default(c)
@@ -110,115 +116,165 @@ func (m *Module) CallbackGETHandler(c *gin.Context) {
 		return
 	}
 
-	user, errWithCode := m.parseUserFromClaims(c.Request.Context(), claims, net.IP(c.ClientIP()), app.ID)
+	user, errWithCode := m.fetchUserForClaims(c.Request.Context(), claims, net.IP(c.ClientIP()), app.ID)
 	if errWithCode != nil {
 		m.clearSession(s)
 		api.ErrorHandler(c, errWithCode, m.processor.InstanceGet)
 		return
 	}
+	if user == nil {
+		// no user exists yet - let's ask them for their preferred username
+		instance, errWithCode := m.processor.InstanceGet(c.Request.Context(), config.GetHost())
+		if errWithCode != nil {
+			api.ErrorHandler(c, errWithCode, m.processor.InstanceGet)
+			return
+		}
 
+		// store the claims in the session - that way we know the user is authenticated when processing the form later
+		s.Set(sessionClaims, claims)
+		s.Set(sessionAppID, app.ID)
+		if err := s.Save(); err != nil {
+			m.clearSession(s)
+			api.ErrorHandler(c, gtserror.NewErrorInternalError(err), m.processor.InstanceGet)
+			return
+		}
+		c.HTML(http.StatusOK, "finalize.tmpl", gin.H{
+			"instance":          instance,
+			"name":              claims.Name,
+			"preferredUsername": claims.PreferredUsername,
+		})
+		return
+	}
 	s.Set(sessionUserID, user.ID)
 	if err := s.Save(); err != nil {
 		m.clearSession(s)
 		api.ErrorHandler(c, gtserror.NewErrorInternalError(err), m.processor.InstanceGet)
 		return
 	}
-
 	c.Redirect(http.StatusFound, OauthAuthorizePath)
 }
 
-func (m *Module) parseUserFromClaims(ctx context.Context, claims *oidc.Claims, ip net.IP, appID string) (*gtsmodel.User, gtserror.WithCode) {
+// FinalizePOSTHandler registers the user after additional data has been provided
-	if claims.Email == "" {
+func (m *Module) FinalizePOSTHandler(c *gin.Context) {
-		err := errors.New("no email returned in claims")
+	s := sessions.Default(c)
-		return nil, gtserror.NewErrorBadRequest(err, err.Error())
+
+	form := &extraInfo{}
+	if err := c.ShouldBind(form); err != nil {
+		m.clearSession(s)
+		api.ErrorHandler(c, gtserror.NewErrorBadRequest(err, oauth.HelpfulAdvice), m.processor.InstanceGet)
+		return
 	}
 
-	// see if we already have a user for this email address
+	// since we have multiple possible validation error, `validationError` is a shorthand for rendering them
-	// if so, we don't need to continue + create one
+	validationError := func(err error) {
-	user, err := m.db.GetUserByEmailAddress(ctx, claims.Email)
+		instance, errWithCode := m.processor.InstanceGet(c.Request.Context(), config.GetHost())
+		if errWithCode != nil {
+			api.ErrorHandler(c, errWithCode, m.processor.InstanceGet)
+			return
+		}
+		c.HTML(http.StatusOK, "finalize.tmpl", gin.H{
+			"instance":          instance,
+			"name":              form.Name,
+			"preferredUsername": form.Username,
+			"error":             err,
+		})
+	}
+
+	// check if the username conforms to the spec
+	if err := validate.Username(form.Username); err != nil {
+		validationError(err)
+		return
+	}
+
+	// see if the username is still available
+	usernameAvailable, err := m.db.IsUsernameAvailable(c.Request.Context(), form.Username)
+	if err != nil {
+		api.ErrorHandler(c, gtserror.NewErrorBadRequest(err, oauth.HelpfulAdvice), m.processor.InstanceGet)
+		return
+	}
+	if !usernameAvailable {
+		validationError(fmt.Errorf("Username %s is already taken", form.Username))
+		return
+	}
+
+	// retrieve the information previously set by the oidc logic
+	appID, ok := s.Get(sessionAppID).(string)
+	if !ok {
+		err := fmt.Errorf("key %s was not found in session", sessionAppID)
+		api.ErrorHandler(c, gtserror.NewErrorBadRequest(err, oauth.HelpfulAdvice), m.processor.InstanceGet)
+		return
+	}
+
+	// retrieve the claims returned by the IDP. Having this present means that we previously already verified these claims
+	claims, ok := s.Get(sessionClaims).(*oidc.Claims)
+	if !ok {
+		err := fmt.Errorf("key %s was not found in session", sessionClaims)
+		api.ErrorHandler(c, gtserror.NewErrorBadRequest(err, oauth.HelpfulAdvice), m.processor.InstanceGet)
+		return
+	}
+
+	// we're now ready to actually create the user
+	user, errWithCode := m.createUserFromOIDC(c.Request.Context(), claims, form, net.IP(c.ClientIP()), appID)
+	if errWithCode != nil {
+		m.clearSession(s)
+		api.ErrorHandler(c, errWithCode, m.processor.InstanceGet)
+		return
+	}
+	s.Delete(sessionClaims)
+	s.Delete(sessionAppID)
+	s.Set(sessionUserID, user.ID)
+	if err := s.Save(); err != nil {
+		m.clearSession(s)
+		api.ErrorHandler(c, gtserror.NewErrorInternalError(err), m.processor.InstanceGet)
+		return
+	}
+	c.Redirect(http.StatusFound, OauthAuthorizePath)
+}
+
+func (m *Module) fetchUserForClaims(ctx context.Context, claims *oidc.Claims, ip net.IP, appID string) (*gtsmodel.User, gtserror.WithCode) {
+	if claims.Sub == "" {
+		err := errors.New("no sub claim found - is your provider OIDC compliant?")
+		return nil, gtserror.NewErrorBadRequest(err, err.Error())
+	}
+	user, err := m.db.GetUserByExternalID(ctx, claims.Sub)
 	if err == nil {
 		return user, nil
 	}
-
 	if err != db.ErrNoEntries {
+		err := fmt.Errorf("error checking database for externalID %s: %s", claims.Sub, err)
+		return nil, gtserror.NewErrorInternalError(err)
+	}
+	if !config.GetOIDCLinkExisting() {
+		return nil, nil
+	}
+	// fallback to email if we want to link existing users
+	user, err = m.db.GetUserByEmailAddress(ctx, claims.Email)
+	if err == db.ErrNoEntries {
+		return nil, nil
+	} else if err != nil {
 		err := fmt.Errorf("error checking database for email %s: %s", claims.Email, err)
 		return nil, gtserror.NewErrorInternalError(err)
 	}
+	// at this point we have found a matching user but still need to link the newly received external ID
 
-	// maybe we have an unconfirmed user
+	user.ExternalID = claims.Sub
-	err = m.db.GetWhere(ctx, []db.Where{{Key: "unconfirmed_email", Value: claims.Email}}, user)
+	err = m.db.UpdateUser(ctx, user, "external_id")
-	if err == nil {
+	if err != nil {
-		err := fmt.Errorf("user with email address %s is unconfirmed", claims.Email)
+		err := fmt.Errorf("error linking existing user %s: %s", claims.Email, err)
-		return nil, gtserror.NewErrorForbidden(err, err.Error())
-	}
-
-	if err != db.ErrNoEntries {
-		err := fmt.Errorf("error checking database for email %s: %s", claims.Email, err)
 		return nil, gtserror.NewErrorInternalError(err)
 	}
+	return user, nil
+}
 
-	// we don't have a confirmed or unconfirmed user with the claimed email address
+func (m *Module) createUserFromOIDC(ctx context.Context, claims *oidc.Claims, extraInfo *extraInfo, ip net.IP, appID string) (*gtsmodel.User, gtserror.WithCode) {
-	// however, because we trust the OIDC provider, we should now create a user + account with the provided claims
-
 	// check if the email address is available for use; if it's not there's nothing we can so
 	emailAvailable, err := m.db.IsEmailAvailable(ctx, claims.Email)
 	if err != nil {
 		return nil, gtserror.NewErrorBadRequest(err)
 	}
 	if !emailAvailable {
-		return nil, gtserror.NewErrorConflict(fmt.Errorf("email address %s is not available", claims.Email))
+		help := "The email address given to us by your authentication provider already exists in our records and the server administrator has not enabled account migration"
-	}
+		return nil, gtserror.NewErrorConflict(fmt.Errorf("email address %s is not available", claims.Email), help)
-
-	// now we need a username
-	var username string
-
-	// make sure claims.Name is defined since we'll be using that for the username
-	if claims.Name == "" {
-		err := errors.New("no name returned in claims")
-		return nil, gtserror.NewErrorBadRequest(err, err.Error())
-	}
-
-	// check if we can just use claims.Name as-is
-	if err = validate.Username(claims.Name); err == nil {
-		// the name we have on the claims is already a valid username
-		username = claims.Name
-	} else {
-		// not a valid username so we have to fiddle with it to try to make it valid
-		// first trim leading and trailing whitespace
-		trimmed := strings.TrimSpace(claims.Name)
-		// underscore any spaces in the middle of the name
-		underscored := strings.ReplaceAll(trimmed, " ", "_")
-		// lowercase the whole thing
-		lower := strings.ToLower(underscored)
-		// see if this is valid....
-		if err := validate.Username(lower); err != nil {
-			err := fmt.Errorf("couldn't parse a valid username from claims.Name value of %s: %s", claims.Name, err)
-			return nil, gtserror.NewErrorBadRequest(err, err.Error())
-		}
-		// we managed to get a valid username
-		username = lower
-	}
-
-	var iString string
-	var found bool
-	// if the username isn't available we need to iterate on it until we find one that is
-	// we should try to do this in a predictable way so we just keep iterating i by one and trying
-	// the username with that number on the end
-	//
-	// note that for the first iteration, iString is still "" when the check is made, so our first choice
-	// is still the raw username with no integer stuck on the end
-	for i := 1; !found; i++ {
-		usernameAvailable, err := m.db.IsUsernameAvailable(ctx, username+iString)
-		if err != nil {
-			return nil, gtserror.NewErrorInternalError(err)
-		}
-		if usernameAvailable {
-			// no error so we've found a username that works
-			found = true
-			username += iString
-			continue
-		}
-		iString = strconv.Itoa(i)
 	}
 
 	// check if the user is in any recognised admin groups
@@ -246,7 +302,7 @@ func (m *Module) parseUserFromClaims(ctx context.Context, claims *oidc.Claims, i
 	emailVerified := true
 
 	// create the user! this will also create an account and store it in the database so we don't need to do that here
-	user, err = m.db.NewSignup(ctx, username, "", requireApproval, claims.Email, password, ip, "", appID, emailVerified, admin)
+	user, err := m.db.NewSignup(ctx, extraInfo.Username, "", requireApproval, claims.Email, password, ip, "", appID, emailVerified, claims.Sub, admin)
 	if err != nil {
 		return nil, gtserror.NewErrorInternalError(err)
 	}

internal/config/config.go

@@ -114,6 +114,7 @@ type Configuration struct {
 	OIDCClientID         string   `name:"oidc-client-id" usage:"ClientID of GoToSocial, as registered with the OIDC provider."`
 	OIDCClientSecret     string   `name:"oidc-client-secret" usage:"ClientSecret of GoToSocial, as registered with the OIDC provider."`
 	OIDCScopes           []string `name:"oidc-scopes" usage:"OIDC scopes."`
+	OIDCLinkExisting     bool     `name:"oidc-link-existing" usage:"link existing user accounts to OIDC logins based on the stored email value"`
 
 	SMTPHost     string `name:"smtp-host" usage:"Host of the smtp server. Eg., 'smtp.eu.mailgun.org'"`
 	SMTPPort     int    `name:"smtp-port" usage:"Port of the smtp server. Eg., 587"`

internal/config/defaults.go

@@ -87,6 +87,7 @@
 	OIDCClientID:         "",
 	OIDCClientSecret:     "",
 	OIDCScopes:           []string{oidc.ScopeOpenID, "profile", "email", "groups"},
+	OIDCLinkExisting:     false,
 
 	SMTPHost:     "",
 	SMTPPort:     0,

internal/config/helpers.gen.go

@@ -1545,6 +1545,31 @@ func GetOIDCScopes() []string { return global.GetOIDCScopes() }
 // SetOIDCScopes safely sets the value for global configuration 'OIDCScopes' field
 func SetOIDCScopes(v []string) { global.SetOIDCScopes(v) }
 
+// GetOIDCLinkExisting safely fetches the Configuration value for state's 'OIDCLinkExisting' field
+func (st *ConfigState) GetOIDCLinkExisting() (v bool) {
+	st.mutex.Lock()
+	v = st.config.OIDCLinkExisting
+	st.mutex.Unlock()
+	return
+}
+
+// SetOIDCLinkExisting safely sets the Configuration value for state's 'OIDCLinkExisting' field
+func (st *ConfigState) SetOIDCLinkExisting(v bool) {
+	st.mutex.Lock()
+	defer st.mutex.Unlock()
+	st.config.OIDCLinkExisting = v
+	st.reloadToViper()
+}
+
+// OIDCLinkExistingFlag returns the flag name for the 'OIDCLinkExisting' field
+func OIDCLinkExistingFlag() string { return "oidc-link-existing" }
+
+// GetOIDCLinkExisting safely fetches the value for global configuration 'OIDCLinkExisting' field
+func GetOIDCLinkExisting() bool { return global.GetOIDCLinkExisting() }
+
+// SetOIDCLinkExisting safely sets the value for global configuration 'OIDCLinkExisting' field
+func SetOIDCLinkExisting(v bool) { global.SetOIDCLinkExisting(v) }
+
 // GetSMTPHost safely fetches the Configuration value for state's 'SMTPHost' field
 func (st *ConfigState) GetSMTPHost() (v string) {
 	st.mutex.Lock()
@@ -1919,3 +1944,4 @@ func GetAdminMediaPruneDryRun() bool { return global.GetAdminMediaPruneDryRun()
 
 // SetAdminMediaPruneDryRun safely sets the value for global configuration 'AdminMediaPruneDryRun' field
 func SetAdminMediaPruneDryRun(v bool) { global.SetAdminMediaPruneDryRun(v) }
+

internal/db/admin.go

@@ -40,7 +40,7 @@ type Admin interface {
 
 	// NewSignup creates a new user in the database with the given parameters.
 	// By the time this function is called, it should be assumed that all the parameters have passed validation!
-	NewSignup(ctx context.Context, username string, reason string, requireApproval bool, email string, password string, signUpIP net.IP, locale string, appID string, emailVerified bool, admin bool) (*gtsmodel.User, Error)
+	NewSignup(ctx context.Context, username string, reason string, requireApproval bool, email string, password string, signUpIP net.IP, locale string, appID string, emailVerified bool, externalID string, admin bool) (*gtsmodel.User, Error)
 
 	// CreateInstanceAccount creates an account in the database with the same username as the instance host value.
 	// Ie., if the instance is hosted at 'example.org' the instance user will have a username of 'example.org'.

internal/db/bundb/admin.go

@@ -90,7 +90,7 @@ func (a *adminDB) IsEmailAvailable(ctx context.Context, email string) (bool, db.
 	return a.conn.NotExists(ctx, q)
 }
 
-func (a *adminDB) NewSignup(ctx context.Context, username string, reason string, requireApproval bool, email string, password string, signUpIP net.IP, locale string, appID string, emailVerified bool, admin bool) (*gtsmodel.User, db.Error) {
+func (a *adminDB) NewSignup(ctx context.Context, username string, reason string, requireApproval bool, email string, password string, signUpIP net.IP, locale string, appID string, emailVerified bool, externalID string, admin bool) (*gtsmodel.User, db.Error) {
 	key, err := rsa.GenerateKey(rand.Reader, rsaKeyBits)
 	if err != nil {
 		log.Errorf("error creating new rsa key: %s", err)
@@ -169,6 +169,7 @@ func (a *adminDB) NewSignup(ctx context.Context, username string, reason string,
 		UnconfirmedEmail:       email,
 		CreatedByApplicationID: appID,
 		Approved:               &approved,
+		ExternalID:             externalID,
 	}
 
 	if emailVerified {

internal/db/bundb/migrations/20221103203553_add_external_id.go

@@ -0,0 +1,46 @@
+/*
+   GoToSocial
+   Copyright (C) 2021-2022 GoToSocial Authors admin@gotosocial.org
+
+   This program is free software: you can redistribute it and/or modify
+   it under the terms of the GNU Affero General Public License as published by
+   the Free Software Foundation, either version 3 of the License, or
+   (at your option) any later version.
+
+   This program is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+   GNU Affero General Public License for more details.
+
+   You should have received a copy of the GNU Affero General Public License
+   along with this program.  If not, see <http://www.gnu.org/licenses/>.
+*/
+
+package migrations
+
+import (
+	"context"
+	"strings"
+
+	"github.com/uptrace/bun"
+)
+
+func init() {
+	up := func(ctx context.Context, db *bun.DB) error {
+		_, err := db.ExecContext(ctx, "ALTER TABLE ? ADD COLUMN ? TEXT", bun.Ident("users"), bun.Ident("external_id"))
+		if err != nil && !(strings.Contains(err.Error(), "already exists") || strings.Contains(err.Error(), "duplicate column name") || strings.Contains(err.Error(), "SQLSTATE 42701")) {
+			return err
+		}
+		return nil
+	}
+
+	down := func(ctx context.Context, db *bun.DB) error {
+		return db.RunInTx(ctx, nil, func(ctx context.Context, tx bun.Tx) error {
+			return nil
+		})
+	}
+
+	if err := Migrations.Register(up, down); err != nil {
+		panic(err)
+	}
+}

internal/db/bundb/user.go

@@ -40,6 +40,7 @@ func (u *userDB) init() {
 		{Name: "AccountID"},
 		{Name: "Email"},
 		{Name: "ConfirmationToken"},
+		{Name: "ExternalID"},
 	}, func(u1 *gtsmodel.User) *gtsmodel.User {
 		u2 := new(gtsmodel.User)
 		*u2 = *u1
@@ -104,6 +105,24 @@ func (u *userDB) GetUserByEmailAddress(ctx context.Context, emailAddress string)
 		return &user, nil
 	}, emailAddress)
 }
+func (u *userDB) GetUserByExternalID(ctx context.Context, id string) (*gtsmodel.User, db.Error) {
+
+	return u.cache.Load("ExternalID", func() (*gtsmodel.User, error) {
+		var user gtsmodel.User
+
+		q := u.conn.
+			NewSelect().
+			Model(&user).
+			Relation("Account").
+			Where("? = ?", bun.Ident("user.external_id"), id)
+
+		if err := q.Scan(ctx); err != nil {
+			return nil, u.conn.ProcessError(err)
+		}
+
+		return &user, nil
+	}, id)
+}
 
 func (u *userDB) GetUserByConfirmationToken(ctx context.Context, confirmationToken string) (*gtsmodel.User, db.Error) {
 	return u.cache.Load("ConfirmationToken", func() (*gtsmodel.User, error) {

internal/db/user.go

@@ -32,6 +32,8 @@ type User interface {
 	GetUserByAccountID(ctx context.Context, accountID string) (*gtsmodel.User, Error)
 	// GetUserByID returns one user with the given email address, or an error if something goes wrong.
 	GetUserByEmailAddress(ctx context.Context, emailAddress string) (*gtsmodel.User, Error)
+	// GetUserByExternalID returns one user with the given external id, or an error if something goes wrong.
+	GetUserByExternalID(ctx context.Context, id string) (*gtsmodel.User, Error)
 	// GetUserByConfirmationToken returns one user by its confirmation token, or an error if something goes wrong.
 	GetUserByConfirmationToken(ctx context.Context, confirmationToken string) (*gtsmodel.User, Error)
 	// PutUser will attempt to place user in the database

internal/gtsmodel/user.go

@@ -56,4 +56,5 @@ type User struct {
 	Approved               *bool        `validate:"-" bun:",nullzero,notnull,default:false"`                             // Has this user been approved by a moderator?
 	ResetPasswordToken     string       `validate:"required_with=ResetPasswordSentAt" bun:",nullzero"`                   // The generated token that the user can use to reset their password
 	ResetPasswordSentAt    time.Time    `validate:"required_with=ResetPasswordToken" bun:"type:timestamptz,nullzero"`    // When did we email the user their reset-password email?
+	ExternalID             string       `validate:"-" bun:",nullzero,unique"`                                            // If the login for the user is managed externally (e.g OIDC), we need to keep a stable reference to the external object (e.g OIDC sub claim)
 }

internal/oidc/claims.go

@@ -18,10 +18,18 @@
 
 package oidc
 
+import "encoding/gob"
+
 // Claims represents claims as found in an id_token returned from an OIDC flow.
 type Claims struct {
-	Email         string   `json:"email"`
+	Sub               string   `json:"sub"`
-	EmailVerified bool     `json:"email_verified"`
+	Email             string   `json:"email"`
-	Groups        []string `json:"groups"`
+	EmailVerified     bool     `json:"email_verified"`
-	Name          string   `json:"name"`
+	Groups            []string `json:"groups"`
+	Name              string   `json:"name"`
+	PreferredUsername string   `json:"preferred_username"`
+}
+
+func init() {
+	gob.Register(&Claims{})
 }

internal/processing/account/create.go

@@ -60,7 +60,7 @@ func (p *processor) Create(ctx context.Context, applicationToken oauth2.TokenInf
 	}
 
 	log.Trace("creating new username and account")
-	user, err := p.db.NewSignup(ctx, form.Username, text.SanitizePlaintext(reason), approvalRequired, form.Email, form.Password, form.IP, form.Locale, application.ID, false, false)
+	user, err := p.db.NewSignup(ctx, form.Username, text.SanitizePlaintext(reason), approvalRequired, form.Email, form.Password, form.IP, form.Locale, application.ID, false, "", false)
 	if err != nil {
 		return nil, gtserror.NewErrorInternalError(fmt.Errorf("error creating new signup in the database: %s", err))
 	}

test/envparsing.sh

@@ -2,7 +2,7 @@
 
 set -eu
 
-EXPECT='{"account-domain":"peepee","accounts-allow-custom-css":true,"accounts-approval-required":false,"accounts-reason-required":false,"accounts-registration-open":true,"advanced-cookies-samesite":"strict","advanced-rate-limit-requests":6969,"application-name":"gts","bind-address":"127.0.0.1","config-path":"internal/config/testdata/test.yaml","db-address":":memory:","db-database":"gotosocial_prod","db-password":"hunter2","db-port":6969,"db-tls-ca-cert":"","db-tls-mode":"disable","db-type":"sqlite","db-user":"sex-haver","dry-run":false,"email":"","host":"example.com","instance-deliver-to-shared-inboxes":false,"instance-expose-peers":true,"instance-expose-public-timeline":true,"instance-expose-suspended":true,"landing-page-user":"admin","letsencrypt-cert-dir":"/gotosocial/storage/certs","letsencrypt-email-address":"","letsencrypt-enabled":true,"letsencrypt-port":80,"log-db-queries":true,"log-level":"info","media-description-max-chars":5000,"media-description-min-chars":69,"media-emoji-local-max-size":420,"media-emoji-remote-max-size":420,"media-image-max-size":420,"media-remote-cache-days":30,"media-video-max-size":420,"oidc-client-id":"1234","oidc-client-secret":"shhhh its a secret","oidc-enabled":true,"oidc-idp-name":"sex-haver","oidc-issuer":"whoknows","oidc-scopes":["read","write"],"oidc-skip-verification":true,"password":"","path":"","port":6969,"protocol":"http","smtp-from":"queen.rip.in.piss@terfisland.org","smtp-host":"example.com","smtp-password":"hunter2","smtp-port":4269,"smtp-username":"sex-haver","software-version":"","statuses-cw-max-chars":420,"statuses-max-chars":69,"statuses-media-max-files":1,"statuses-poll-max-options":1,"statuses-poll-option-max-chars":50,"storage-backend":"local","storage-local-base-path":"/root/store","storage-s3-access-key":"minio","storage-s3-bucket":"gts","storage-s3-endpoint":"localhost:9000","storage-s3-proxy":true,"storage-s3-secret-key":"miniostorage","storage-s3-use-ssl":false,"syslog-address":"127.0.0.1:6969","syslog-enabled":true,"syslog-protocol":"udp","trusted-proxies":["127.0.0.1/32","docker.host.local"],"username":"","web-asset-base-dir":"/root","web-template-base-dir":"/root"}'
+EXPECT='{"account-domain":"peepee","accounts-allow-custom-css":true,"accounts-approval-required":false,"accounts-reason-required":false,"accounts-registration-open":true,"advanced-cookies-samesite":"strict","advanced-rate-limit-requests":6969,"application-name":"gts","bind-address":"127.0.0.1","config-path":"internal/config/testdata/test.yaml","db-address":":memory:","db-database":"gotosocial_prod","db-password":"hunter2","db-port":6969,"db-tls-ca-cert":"","db-tls-mode":"disable","db-type":"sqlite","db-user":"sex-haver","dry-run":false,"email":"","host":"example.com","instance-deliver-to-shared-inboxes":false,"instance-expose-peers":true,"instance-expose-public-timeline":true,"instance-expose-suspended":true,"landing-page-user":"admin","letsencrypt-cert-dir":"/gotosocial/storage/certs","letsencrypt-email-address":"","letsencrypt-enabled":true,"letsencrypt-port":80,"log-db-queries":true,"log-level":"info","media-description-max-chars":5000,"media-description-min-chars":69,"media-emoji-local-max-size":420,"media-emoji-remote-max-size":420,"media-image-max-size":420,"media-remote-cache-days":30,"media-video-max-size":420,"oidc-client-id":"1234","oidc-client-secret":"shhhh its a secret","oidc-enabled":true,"oidc-idp-name":"sex-haver","oidc-issuer":"whoknows","oidc-link-existing":true,"oidc-scopes":["read","write"],"oidc-skip-verification":true,"password":"","path":"","port":6969,"protocol":"http","smtp-from":"queen.rip.in.piss@terfisland.org","smtp-host":"example.com","smtp-password":"hunter2","smtp-port":4269,"smtp-username":"sex-haver","software-version":"","statuses-cw-max-chars":420,"statuses-max-chars":69,"statuses-media-max-files":1,"statuses-poll-max-options":1,"statuses-poll-option-max-chars":50,"storage-backend":"local","storage-local-base-path":"/root/store","storage-s3-access-key":"minio","storage-s3-bucket":"gts","storage-s3-endpoint":"localhost:9000","storage-s3-proxy":true,"storage-s3-secret-key":"miniostorage","storage-s3-use-ssl":false,"syslog-address":"127.0.0.1:6969","syslog-enabled":true,"syslog-protocol":"udp","trusted-proxies":["127.0.0.1/32","docker.host.local"],"username":"","web-asset-base-dir":"/root","web-template-base-dir":"/root"}'
 
 # Set all the environment variables to 
 # ensure that these are parsed without panic
@@ -65,6 +65,7 @@ GTS_OIDC_ISSUER='whoknows' \
 GTS_OIDC_CLIENT_ID='1234' \
 GTS_OIDC_CLIENT_SECRET='shhhh its a secret' \
 GTS_OIDC_SCOPES='read,write' \
+GTS_OIDC_LINK_EXISTING=true \
 GTS_SMTP_HOST='example.com' \
 GTS_SMTP_PORT=4269 \
 GTS_SMTP_USERNAME='sex-haver' \

testrig/config.go

@@ -94,6 +94,7 @@ func InitTestConfig() {
 	OIDCClientID:         "",
 	OIDCClientSecret:     "",
 	OIDCScopes:           []string{oidc.ScopeOpenID, "profile", "email", "groups"},
+	OIDCLinkExisting:     false,
 
 	SMTPHost:     "",
 	SMTPPort:     0,

web/source/css/base.css

@@ -379,4 +379,18 @@ footer {
 
 .monospace {
 	font-family: monospace;
-}
+}
+
+.callout {
+	margin: 1.5rem 0;
+	border: .05rem solid $border-accent;
+	border-radius: .2rem;
+	padding: 0 .6rem .6rem;
+	.callout-title {
+		margin: 0 -.6rem;
+		padding: .6rem;
+		font-weight: bold;
+		background-color: $border-accent;
+		color: $gray1;
+	}
+}

web/template/finalize.tmpl

@@ -0,0 +1,31 @@
+{{ template "header.tmpl" .}}
+    <main>
+        <form action="/oauth/finalize" method="POST">
+            <h1>Hi {{.name}}!</h1>
+            <p>
+              You are about to sign-up to {{ .instance.Title }} (<code>{{ .instance.URI }}</code>)
+              <br>
+              To ensure the best experience for you, we need you to provide some additional details.
+            </p>
+            {{if .error}}
+              <section class="error">
+                <span>❌</span> <pre>{{.error}}</pre>
+              </section>
+            {{end}}
+            <div class="callout">
+              <p class="callout-title">Important</p>
+              <p>Due to the way the ActivityPub standard works, you <strong>cannot</strong> change your username after it has been set.</p>
+            </div>
+            <div class="labelinput">
+                <label for="username">Username <small>(must contain only lowercase letters, numbers, and underscores)</small></label>
+                <input type="text"
+                       class="form-control"
+                       name="username"
+                       required
+                       placeholder="Please enter your desired username" value="{{ .preferredUsername }}">
+            </div>
+            <input type="hidden" name="name" value="{{ .name }}">
+            <button type="submit" style="width: 100%; margin-top: 1rem;" class="btn btn-success">Submit</button>
+        </form>
+    </main>
+{{ template "footer.tmpl" .}}