Commit graph

1337 commits

Author SHA1 Message Date
kim 6c9d8e78eb
[feature] status refetch support (#1690)
* revamp http client to not limit requests, instead use sender worker

Signed-off-by: kim <grufwub@gmail.com>

* remove separate sender worker pool, spawn 2*GOMAXPROCS batch senders each time, no need for transport cache sweeping

Signed-off-by: kim <grufwub@gmail.com>

* improve batch senders to keep popping recipients until remote URL found

Signed-off-by: kim <grufwub@gmail.com>

* fix recipient looping issue

Signed-off-by: kim <grufwub@gmail.com>

* move request id ctx key to gtscontext, finish filling out more code comments, add basic support for not logging client IP

Signed-off-by: kim <grufwub@gmail.com>

* first draft of status refetching logic

Signed-off-by: kim <grufwub@gmail.com>

* fix testrig to use new federation alloc func signature

Signed-off-by: kim <grufwub@gmail.com>

* fix log format directive

Signed-off-by: kim <grufwub@gmail.com>

* add status fetched_at migration

Signed-off-by: kim <grufwub@gmail.com>

* remove unused / unchecked for error types

Signed-off-by: kim <grufwub@gmail.com>

* add back the used type...

Signed-off-by: kim <grufwub@gmail.com>

* add separate internal getStatus() function for derefThread() that doesn't recurse

Signed-off-by: kim <grufwub@gmail.com>

* improved mention and media attachment error handling

Signed-off-by: kim <grufwub@gmail.com>

* fix log and error format directives

Signed-off-by: kim <grufwub@gmail.com>

* update account deref to match status deref changes

Signed-off-by: kim <grufwub@gmail.com>

* very small code formatting change to make things clearer

Signed-off-by: kim <grufwub@gmail.com>

* add more code comments

Signed-off-by: kim <grufwub@gmail.com>

* improved code commenting

Signed-off-by: kim <grufwub@gmail.com>

* only check for required further derefs if needed

Signed-off-by: kim <grufwub@gmail.com>

* improved cache invalidation

Signed-off-by: kim <grufwub@gmail.com>

* tweak cache restarting to use a (very small) backoff

Signed-off-by: kim <grufwub@gmail.com>

* small readability changes and fixes

Signed-off-by: kim <grufwub@gmail.com>

* fix account sync issues

Signed-off-by: kim <grufwub@gmail.com>

* fix merge conflicts + update account enrichment to accept already-passed accountable

Signed-off-by: kim <grufwub@gmail.com>

* remove secondary function declaration

Signed-off-by: kim <grufwub@gmail.com>

* normalise dereferencer get status / account behaviour, fix remaining tests

Signed-off-by: kim <grufwub@gmail.com>

* fix remaining rebase conflicts, finish commenting code

Signed-off-by: kim <grufwub@gmail.com>

* appease the linter

Signed-off-by: kim <grufwub@gmail.com>

* add source file header

Signed-off-by: kim <grufwub@gmail.com>

* update to use TIMESTAMPTZ column type instead of just TIMESTAMP

Signed-off-by: kim <grufwub@gmail.com>

* don't pass in 'updated_at' to UpdateEmoji()

Signed-off-by: kim <grufwub@gmail.com>

* use new ap.Resolve{Account,Status}able() functions

Signed-off-by: kim <grufwub@gmail.com>

* remove the somewhat confusing rescoping of the same variable names

Signed-off-by: kim <grufwub@gmail.com>

* update migration file name, improved database delete error returns

Signed-off-by: kim <grufwub@gmail.com>

* formatting

Signed-off-by: kim <grufwub@gmail.com>

* improved multi-delete database functions to minimise DB calls

Signed-off-by: kim <grufwub@gmail.com>

* remove unused type

Signed-off-by: kim <grufwub@gmail.com>

* fix delete statements

Signed-off-by: kim <grufwub@gmail.com>

---------

Signed-off-by: kim <grufwub@gmail.com>
2023-05-12 11:15:54 +02:00
Natsu Kagami ba5a464ca5
[chore] Prefer JSON errors in API endpoints (#1766)
* Default to JSON over HTML for error handling

* Change the default error display for web endpoints to html
2023-05-12 10:16:41 +02:00
f0x52 265cc32516
use more semantic headers for profile page (#1765) 2023-05-11 20:36:14 +02:00
f0x52 9cc9ffc5a7
[frontend] Profiles with fields & more (#1764)
* redesign status template

* separate index page styling

* redesign profile template

* fix header styling/wrapping

* remove old spoiler js

* fix status cw button wrapping

* fix status info variables

* profile responsiveness, accessibility tweaks

* fix variable use, mobile

* remove duplicate id's

* rss icon, fix indent

* fix toot border-radius

* fix toot spacing

* emojify and html profile fields

* refactor (sensitive) media rendering

* plaintext profile fields

* bundle plyr icon svg

* only pause video when switching photoswipe slides

* yarn upgrade

* profile fields formatting

* replace uglifyify with @browserify updated fork

* fix profile field templating (yet again)

* fix React classes

* testrig: add testing profile field for admin user

* fix sensitive media interactions

* Revert "testrig: add testing profile field for admin user"

This reverts commit 80490c183e.

* settings interface wrapping

* fix reported toot styling

* add role to profile sr-only text

* comment fallback rule

* remove currently unused image description lacking indicator
2023-05-11 17:46:32 +02:00
Dominik Süß 6392e00653
feat: initial tracing support (#1623) 2023-05-09 18:19:48 +01:00
tobi 878ed48de3
[bugfix] Don't try to get user when serializing local instance account (#1757) 2023-05-09 16:05:35 +01:00
kim 11e843a273
[bugfix] fix possible domain blockcache nil ptr + add debug String() func (#1755) 2023-05-09 15:18:51 +01:00
kim 8275d70e38
[bugfix] update go-cache version to support multi-keying (#1756)
* update go-cache version to support multi-keying

Signed-off-by: kim <grufwub@gmail.com>

* improved cache invalidation

Signed-off-by: kim <grufwub@gmail.com>

---------

Signed-off-by: kim <grufwub@gmail.com>
2023-05-09 16:17:43 +02:00
tobi 65cd1acbdf
[chore] tidy + update StatusToAPIStatus (#1754) 2023-05-09 13:25:48 +02:00
tobi 0e29f1f5bb
[feature] Enable federation in/out of profile PropertyValue fields (#1722)
Co-authored-by: kim <grufwub@gmail.com>
Co-authored-by: kim <89579420+NyaaaWhatsUpDoc@users.noreply.github.com>
2023-05-09 11:16:10 +01:00
tobi cbb9e2d3f0
[chore/performance] Make sender multiplier configurable (#1750) 2023-05-08 18:03:38 +01:00
tobi 37b4d9d179
[bugfix] Punycode fixes (#1743)
Co-authored-by: kim <grufwub@gmail.com>
Co-authored-by: kim <89579420+NyaaaWhatsUpDoc@users.noreply.github.com>
2023-05-07 18:53:21 +01:00
Daenney b7dd32da42
[bugfix] Return languages in api/v1/instance (#1741)
It turns out that in Masto v2.3.0 the languages key was added to the V1
Instance and that it's effectively mandatory. Though in GtS we don't
really have this concept yet, some apps will explode if the languages
key is missing altogether. So at least return the empty array on V1 too
in the hopes that it makes things work well enough.

For history's sake, you can see the attributes that will get serialised
in
f877aa9d70/app/serializers/rest/v1/instance_serializer.rb (L6-L9).
Because the attribute does not have a conditional defined for it,
there isn't a filter that optionally omits it, or a def languages to
modify the behaviour the attribute is effectively always included and
serialised.

Fixes: #1662
2023-05-06 22:38:19 +02:00
Daenney 6d138588d8
[feature] Implement the preferences client API (#1740)
This adds the preferences endpoint to our Mastodon Client API
implementation. It's a read-only endpoint that returns a number of
user preferences. Applications can query these settings when logging in
a user (for the first time) to configure themselves.
2023-05-06 17:42:58 +02:00
tobi 9b4b4d4eb8
[chore] update readthedocs deps, use conda (#1742)
* [chore] update readthedocs deps, use conda

* work this time thx
2023-05-06 17:33:37 +02:00
tobi 5027d0ced2
[bugfix] Serve correct 'application/jrd+json' content type for webfinger requests (#1738)
* [bugfix] Return `application/jrd+json` from webfinger queries

* update finger req content-type
2023-05-04 12:28:50 +02:00
tobi 4a012acd52
[bugfix] Rework notifs to use min_id for paging up (#1734) 2023-05-04 12:27:24 +02:00
tobi a6ec2a5bc2
[bugfix] Fix invalid og:description on account w/ empty note (#1733) 2023-05-03 16:18:34 +02:00
Daenney c41c1f90a6
[chore] Hide vendor from diffs (#1729)
Linguist already classifies vendor/ as vendored files, meaning they
don't count towards things like a repo language stat, but are still
shows in diffs.

By classifying them as generated instead they're not only excluded from
langauge stats but also hidden in diffs.
2023-05-02 21:11:41 +01:00
kim 3ff1391a9d
[performance] replace domain block cache with an in-memory radix trie (#1714)
* replace domain block cache with an in-memory radix tree

Signed-off-by: kim <grufwub@gmail.com>

* fix domain block cache init

Signed-off-by: kim <grufwub@gmail.com>

---------

Signed-off-by: kim <grufwub@gmail.com>
2023-05-01 12:36:46 +02:00
dependabot[bot] 66f09a8d93
[chore]: Bump modernc.org/sqlite from 1.22.0 to 1.22.1 (#1726)
Bumps [modernc.org/sqlite](https://gitlab.com/cznic/sqlite) from 1.22.0 to 1.22.1.
- [Release notes](https://gitlab.com/cznic/sqlite/tags)
- [Commits](https://gitlab.com/cznic/sqlite/compare/v1.22.0...v1.22.1)

---
updated-dependencies:
- dependency-name: modernc.org/sqlite
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-05-01 11:03:30 +02:00
dependabot[bot] e9de7bec15
[chore]: Bump github.com/go-playground/validator/v10 (#1724)
Bumps [github.com/go-playground/validator/v10](https://github.com/go-playground/validator) from 10.12.0 to 10.13.0.
- [Release notes](https://github.com/go-playground/validator/releases)
- [Commits](https://github.com/go-playground/validator/compare/v10.12.0...v10.13.0)

---
updated-dependencies:
- dependency-name: github.com/go-playground/validator/v10
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-05-01 11:02:44 +02:00
dependabot[bot] ab7f518f57
[chore]: Bump github.com/KimMachineGun/automemlimit from 0.2.5 to 0.2.6 (#1723)
Bumps [github.com/KimMachineGun/automemlimit](https://github.com/KimMachineGun/automemlimit) from 0.2.5 to 0.2.6.
- [Release notes](https://github.com/KimMachineGun/automemlimit/releases)
- [Commits](https://github.com/KimMachineGun/automemlimit/compare/v0.2.5...v0.2.6)

---
updated-dependencies:
- dependency-name: github.com/KimMachineGun/automemlimit
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-05-01 11:01:36 +02:00
dependabot[bot] 5904e3b4ee
[chore]: Bump github.com/miekg/dns from 1.1.53 to 1.1.54 (#1727)
Bumps [github.com/miekg/dns](https://github.com/miekg/dns) from 1.1.53 to 1.1.54.
- [Release notes](https://github.com/miekg/dns/releases)
- [Changelog](https://github.com/miekg/dns/blob/master/Makefile.release)
- [Commits](https://github.com/miekg/dns/compare/v1.1.53...v1.1.54)

---
updated-dependencies:
- dependency-name: github.com/miekg/dns
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-05-01 11:00:41 +02:00
kim ead286a67b
[bugfix] tweak httpclient error handling again ... (#1721)
* check for tls, x509 errors using string.Contains() since crypto/tls sucks

Signed-off-by: kim <grufwub@gmail.com>

* use 2* maxprocs

Signed-off-by: kim <grufwub@gmail.com>

---------

Signed-off-by: kim <grufwub@gmail.com>
2023-04-30 10:11:18 +02:00
kim 68b91d2128
[performance] tweak http client error handling (#1718)
* update errors library, check for more TLS type error in http client

Signed-off-by: kim <grufwub@gmail.com>

* bump cache library version to match errors library

Signed-off-by: kim <grufwub@gmail.com>

---------

Signed-off-by: kim <grufwub@gmail.com>
2023-04-29 18:44:20 +02:00
tobi 8b1e2288d8
[feature] Add GET endpoint for single notification (#1719) 2023-04-29 17:29:51 +01:00
Mal Hancock fdd2487cfb
[bugfix] add From to email header (#1717)
* add From to email header

* update tests
2023-04-29 11:47:14 +02:00
kim 6a29c5ffd4
[performance] improved request batching (removes need for queueing) (#1687)
* revamp http client to not limit requests, instead use sender worker

Signed-off-by: kim <grufwub@gmail.com>

* remove separate sender worker pool, spawn 2*GOMAXPROCS batch senders each time, no need for transport cache sweeping

Signed-off-by: kim <grufwub@gmail.com>

* improve batch senders to keep popping recipients until remote URL found

Signed-off-by: kim <grufwub@gmail.com>

* fix recipient looping issue

Signed-off-by: kim <grufwub@gmail.com>

* fix missing mutex unlock

Signed-off-by: kim <grufwub@gmail.com>

* move request id ctx key to gtscontext, finish filling out more code comments, add basic support for not logging client IP

Signed-off-by: kim <grufwub@gmail.com>

* slight code reformatting

Signed-off-by: kim <grufwub@gmail.com>

* a whitespace

Signed-off-by: kim <grufwub@gmail.com>

* remove unused code

Signed-off-by: kim <grufwub@gmail.com>

* add missing license headers

Signed-off-by: kim <grufwub@gmail.com>

* fix request backoff calculation

Signed-off-by: kim <grufwub@gmail.com>

---------

Signed-off-by: kim <grufwub@gmail.com>
2023-04-28 17:45:21 +02:00
tobi 6b4f6dc755
[bugfix] Fix remaining mangled URI escaping issues in statuses + accounts (#1712)
* start fiddling with normalize + extract functions

* normalize attachment name (image description)

* NormalizeAccountableSummary

* normalize summary + name
2023-04-26 16:17:22 +01:00
0hlov3 ae9d432f13
[docs] Migrates Chart Location (#1708) 2023-04-24 13:49:44 +02:00
dependabot[bot] 49395f2464
[chore]: Bump modernc.org/sqlite from 1.21.1 to 1.22.0 (#1711)
Bumps [modernc.org/sqlite](https://gitlab.com/cznic/sqlite) from 1.21.1 to 1.22.0.
- [Release notes](https://gitlab.com/cznic/sqlite/tags)
- [Commits](https://gitlab.com/cznic/sqlite/compare/v1.21.1...v1.22.0)

---
updated-dependencies:
- dependency-name: modernc.org/sqlite
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-04-24 13:49:21 +02:00
tobi fd3bf9284d
[bugfix] fix small config typos (#1707) 2023-04-19 14:22:13 +02:00
OniriCorpe f3066a11bf
[chore] Replacing all Pinafore mentions to Semaphore ones (#1689) 2023-04-19 13:50:10 +02:00
dependabot[bot] a3d2774db5
[chore]: Bump codeberg.org/gruf/go-cache/v3 from 3.2.3 to 3.2.5 (#1701)
Bumps codeberg.org/gruf/go-cache/v3 from 3.2.3 to 3.2.5.

---
updated-dependencies:
- dependency-name: codeberg.org/gruf/go-cache/v3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-04-19 13:46:42 +02:00
dependabot[bot] c5f61c456b
[chore]: Bump mvdan.cc/xurls/v2 from 2.4.0 to 2.5.0 (#1702)
Bumps [mvdan.cc/xurls/v2](https://github.com/mvdan/xurls) from 2.4.0 to 2.5.0.
- [Release notes](https://github.com/mvdan/xurls/releases)
- [Commits](https://github.com/mvdan/xurls/compare/v2.4.0...v2.5.0)

---
updated-dependencies:
- dependency-name: mvdan.cc/xurls/v2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-04-19 13:44:20 +02:00
dependabot[bot] ef3004b77f
[chore]: Bump github.com/minio/minio-go/v7 from 7.0.50 to 7.0.52 (#1703)
Bumps [github.com/minio/minio-go/v7](https://github.com/minio/minio-go) from 7.0.50 to 7.0.52.
- [Release notes](https://github.com/minio/minio-go/releases)
- [Commits](https://github.com/minio/minio-go/compare/v7.0.50...v7.0.52)

---
updated-dependencies:
- dependency-name: github.com/minio/minio-go/v7
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-04-19 13:43:42 +02:00
Phil Hagelberg 922bd89f17
[docs] Explain that markdown is enabled on the user settings page. (#1699)
* Explain that markdown is enabled on the user settings page.

The current wording seems to imply that the input formatting can be
set on a per-post basis, which no client currently supports. It's not
clear that you have to go to the user settings page to change it.

* Update FAQ; some of the missing features are not missing any more!
2023-04-19 13:42:47 +02:00
Umar Getagazov cb1f935013
[bugfix] Fix the bookmarks list API endpoint returning an empty array (#1700) 2023-04-19 13:42:00 +02:00
tobi 093cf2ab12
[feature] Receive notification when followed account posts (if desired) (#1680)
* start working on notifs for new posts

* tidy up a bit

* update swagger

* carry over show reblogs + notify from follow req

* test notify on status post

* update column slice

* dedupe update logic + add tests

* fix own boosts not being timelined

* avoid type check, passing unnecessary accounts

* remove unnecessary 'inReplyToID' check

* add a couple todo's for future db functions
2023-04-10 20:56:02 +01:00
dependabot[bot] c01d2f9b44
[chore]: Bump golang.org/x/oauth2 from 0.6.0 to 0.7.0 (#1684)
Bumps [golang.org/x/oauth2](https://github.com/golang/oauth2) from 0.6.0 to 0.7.0.
- [Release notes](https://github.com/golang/oauth2/releases)
- [Commits](https://github.com/golang/oauth2/compare/v0.6.0...v0.7.0)

---
updated-dependencies:
- dependency-name: golang.org/x/oauth2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-04-10 13:51:03 +02:00
dependabot[bot] 1d8f9c6b16
[chore]: Bump golang.org/x/crypto from 0.7.0 to 0.8.0 (#1685)
Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from 0.7.0 to 0.8.0.
- [Release notes](https://github.com/golang/crypto/releases)
- [Commits](https://github.com/golang/crypto/compare/v0.7.0...v0.8.0)

---
updated-dependencies:
- dependency-name: golang.org/x/crypto
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-04-10 12:35:05 +02:00
dependabot[bot] 014d7ac131
[chore]: Bump github.com/spf13/cobra from 1.6.1 to 1.7.0 (#1686)
Bumps [github.com/spf13/cobra](https://github.com/spf13/cobra) from 1.6.1 to 1.7.0.
- [Release notes](https://github.com/spf13/cobra/releases)
- [Commits](https://github.com/spf13/cobra/compare/v1.6.1...v1.7.0)

---
updated-dependencies:
- dependency-name: github.com/spf13/cobra
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-04-10 12:14:21 +02:00
dependabot[bot] 158facf011
[chore]: Bump golang.org/x/image from 0.6.0 to 0.7.0 (#1682)
Bumps [golang.org/x/image](https://github.com/golang/image) from 0.6.0 to 0.7.0.
- [Release notes](https://github.com/golang/image/releases)
- [Commits](https://github.com/golang/image/compare/v0.6.0...v0.7.0)

---
updated-dependencies:
- dependency-name: golang.org/x/image
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-04-10 12:12:46 +02:00
tobi f28ed21343
[bugfix] Fix single author threads not appearing in home timeline (#1679) 2023-04-08 12:56:55 +02:00
tobi fb41d2d48f
[bugfix] warn on maxprocs error instead of failing (#1678)
* [bugfix] warn on maxprocs error instead of failing

* info instead of warn
2023-04-08 11:51:24 +02:00
kim e46323c207
[bugfix] visibility caching and hometimeline (#1675)
* fix visibility caching to use correct type key

Signed-off-by: kim <grufwub@gmail.com>

* check for ID check > max possible ID

Signed-off-by: kim <grufwub@gmail.com>

* update home timeline code to include relevant threads to owner (e.g. between mutuals/follows)

Signed-off-by: kim <grufwub@gmail.com>

---------

Signed-off-by: kim <grufwub@gmail.com>
2023-04-06 17:11:25 +02:00
tobi 3510454768
[bugfix/chore] Refactor timeline code (#1656)
* start poking timelines

* OK yes we're refactoring, but it's nothing like the last time so don't worry

* more fiddling

* update tests, simplify Get

* thanks linter, you're the best, mwah mwah kisses

* do a bit more tidying up

* start buggering about with the prepare function

* fix little oopsie

* start merging lists into 1

* ik heb een heel zwaar leven
nee nee echt waar

* hey it works we did it reddit

* regenerate swagger docs

* tidy up a wee bit

* adjust paging

* fix little error, remove unused functions
2023-04-06 12:43:13 +01:00
tobi c54510bc74
[bugfix] Normalize status content (don't parse status content as IRI) (#1665)
* start fannying about

* finish up Normalize

* tidy up

* pin to tag

* move errors about just a little bit
2023-04-06 12:19:55 +01:00
tobi 4f322f527f
[bugfix] Always serialize orderedItems as array (#1673) 2023-04-06 12:16:53 +01:00