Shiori extension never creates archive #48

crawfordlong · 2023-03-20T13:27:31Z

Data

Shiori version: latest
Database Engine: n/a
Operating system:

server: ubuntu 22.10
client: fedora silverblue 37 + firefox

CLI/Web interface/Web Extension: firefox extension

Describe the bug / actual behavior

Server is set up to create archives of saved bookmarks. Using the web interface, entering a new bookmark correctly creates an archive, pulls a banner image, etc. Using the referenced extension, however, the bookmark creates a plaintext bookmark (url and no banner image). It does not seem to be a matter of waiting for a job to execute, as I left a link for a few hours and it didn't pull the archive. Weirdly, this seems to conflict with another bug report.

Expected behavior

Clicking the shiori browser extension creates a bookmark and content archive.

To Reproduce

Steps to reproduce the behavior:

Described above.

Notes

Add any other context about the problem here.

rmayergfx · 2023-04-29T10:53:29Z

Try this one: https://github.com/go-shiori/shiori-web-ext This is the latest Version 0.8.6 and working fine for me
Firefox Addon ist still not up to date. I hate users just uploading files to extension store and never updates them.

Alistair1231 · 2023-05-21T10:45:54Z

The version I use has the same version number (0.8.6) and I still have this issue.
When I noticed this yesterday and manually tried to create archives/retrieve thumbnails of the bookmarks from the last days/weeks where they were missing, it wouldn't work either.
After I restarted the container, with a docker pull for good measure, I could do manual archives again where they were missing, 5 at a time.

I just added a bookmark using the firefox addon and there is again only the tags and the link. This time the manual archive worked though.

docker log isn't very helpful, but here it is:

docker logs output

time="2023-05-21T10:33:25Z" level=info msg="GET /?search=office" proto=HTTP/1.1 remote="[IP ADDRESS]:36238" reqlen=0 size=5659 status=200
time="2023-05-21T10:33:25Z" level=info msg="GET /js/page/base.js" proto=HTTP/1.1 remote="[IP ADDRESS]:36238" reqlen=0 size=2627 status=200
time="2023-05-21T10:33:25Z" level=info msg="GET /api/bookmarks?keyword=office&tags=&exclude=&page=1" proto=HTTP/1.1 remote="[IP ADDRESS]:36294" reqlen=0 size=486 status=200
time="2023-05-21T10:33:25Z" level=info msg="GET /res/favicon-16x16.png" proto=HTTP/1.1 remote="[IP ADDRESS]:36294" reqlen=0 size=587 status=200
time="2023-05-21T10:33:25Z" level=info msg="GET /api/tags" proto=HTTP/1.1 remote="[IP ADDRESS]:36294" reqlen=0 size=3392 status=200
time="2023-05-21T10:33:50Z" level=info msg="POST /api/logout" proto=HTTP/1.1 remote="[IP ADDRESS]:51424" reqlen=0 size=1 status=200
time="2023-05-21T10:33:50Z" level=info msg="POST /api/login" proto=HTTP/1.1 remote="[IP ADDRESS]:51424" reqlen=53 size=141 status=200
time="2023-05-21T10:34:03Z" level=info msg="POST /api/bookmarks/ext" proto=HTTP/1.1 remote="[IP ADDRESS]:49552" reqlen=305773 size=353289 status=200
time="2023-05-21T10:34:21Z" level=info msg="GET /?search=" proto=HTTP/1.1 remote="[IP ADDRESS]:33432" reqlen=0 size=5659 status=200
time="2023-05-21T10:34:21Z" level=info msg="GET /api/bookmarks?keyword=&tags=&exclude=&page=1" proto=HTTP/1.1 remote="[IP ADDRESS]:33432" reqlen=0 size=15844 status=200
time="2023-05-21T10:34:21Z" level=info msg="GET /api/tags" proto=HTTP/1.1 remote="[IP ADDRESS]:33432" reqlen=0 size=3519 status=200
time="2023-05-21T10:34:21Z" level=info msg="GET /bookmark/139/thumb" proto=HTTP/1.1 remote="[IP ADDRESS]:33446" reqlen=0 size=11976 status=200
time="2023-05-21T10:34:21Z" level=info msg="GET /bookmark/138/thumb" proto=HTTP/1.1 remote="[IP ADDRESS]:33466" reqlen=0 size=6789 status=200
time="2023-05-21T10:34:21Z" level=info msg="GET /bookmark/137/thumb" proto=HTTP/1.1 remote="[IP ADDRESS]:33458" reqlen=0 size=50565 status=200
time="2023-05-21T10:34:21Z" level=info msg="GET /bookmark/136/thumb" proto=HTTP/1.1 remote="[IP ADDRESS]:33474" reqlen=0 size=9434 status=200
time="2023-05-21T10:34:21Z" level=info msg="GET /bookmark/140/thumb" proto=HTTP/1.1 remote="[IP ADDRESS]:33432" reqlen=0 size=133096 status=200
time="2023-05-21T10:34:32Z" level=info msg="POST /api/logout" proto=HTTP/1.1 remote="[IP ADDRESS]:42374" reqlen=0 size=1 status=200
time="2023-05-21T10:34:32Z" level=info msg="POST /api/login" proto=HTTP/1.1 remote="[IP ADDRESS]:42374" reqlen=53 size=141 status=200
time="2023-05-21T10:34:41Z" level=info msg="PUT /api/cache" proto=HTTP/1.1 remote="[IP ADDRESS]:42374" reqlen=55 size=9438 status=200
time="2023-05-21T10:34:41Z" level=info msg="GET /bookmark/165/thumb" proto=HTTP/1.1 remote="[IP ADDRESS]:42374" reqlen=0 size=37714 status=200

I can't find a way to increase the logging level. If you think that is helpful and can tell me how, I can post the output.

The network tab of the addon shows this request when I add something: https://pastebin.com/9QqiAV2w (pastebin becuase very long)
The console of the addon doesn't show any errors

rmayergfx · 2023-05-22T09:14:33Z

No Problem here with synology docker, FF latest release with Extension 0.8.6. Be sure no Adblock or script is blocking. Also set "Create archive by default" on shiori Settings (Webpage) and allow all settings in Extensions.
No problem. Do you have an example URL from an webpage not creating archive?

Alistair1231 · 2023-05-25T09:40:56Z

No Problem here with synology docker, FF latest release with Extension 0.8.6. Be sure no Adblock or script is blocking. Also set "Create archive by default" on shiori Settings (Webpage) and allow all settings in Extensions. No problem. Do you have an example URL from an webpage not creating archive?

If it were an issue with content blocking, shouldn't it then not work manually either? Just to make sure I tried it again without adblock, same result. Here is the request from the addon including response again: https://pastebin.com/6DcFzR7h

Also, in the response of the HTTP request from the addon I see this at the bottom \"createArchive\":false}\n".

\"imageURL\":\"\",\"hasContent\":false,\"hasArchive\":false,\"tags\":[{\"id\":40,\"name\":\"wiki\"},{\"id\":87,\"name\":\"info\"}],\"createArchive\":false}\n"

It seems not to try to make an archive. The reponse also contains the entire HTML of https://en.wikipedia.org/wiki/Main_Page (what I tested it with)

These are my settings

I use this shiori.service file on Ubuntu 22.04 Server:

$  sudo cat /etc/systemd/system/shiori.service
# /etc/systemd/system/shiori.service
[Unit]
Description=Shiori container
After=docker.service

[Service]
Restart=always
ExecStartPre=-/usr/bin/docker rm shiori-1
ExecStart=/usr/bin/docker run \
  --rm \
  --name shiori-1 \
  -p 8080:8080 \
  -v /srv/shiori/shiori-data:/shiori \
   ghcr.io/go-shiori/shiori
ExecStop=/usr/bin/docker stop -t 2 shiori-1

[Install]
WantedBy=multi-user.target

Oh and here is the reverse proxy config (i use caddy)

$ sudo cat /srv/caddy/Caddyfile
{
        acme_dns cloudflare XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
        log {
                output file /srv/caddy/logs/access.log
        }
}

example.com {
        reverse_proxy localhost:8080
}

rmayergfx · 2023-05-25T15:53:51Z

Sorry, no idea what is going wrong on your system. tested the given url from wikipedia an no trouble, all went fine and i have a working archive also:

Alistair1231 · 2023-05-25T22:59:44Z

Is there a log I can provide somewhere? Also, is this correct? The server seems to think everytrhing goes fine, but for some reason it thinks that it shouldn't create an archive, even though that setting is active. At least that's my interpretation.

Here is the request from the addon including response again: https://pastebin.com/6DcFzR7h
Also, in the response of the HTTP request from the addon I see this at the bottom \"createArchive\":false}\n".
\"imageURL\":\"\",\"hasContent\":false,\"hasArchive\":false,\"tags\":[{\"id\":40,\"name\":\"wiki\"},{\"id\":87,\"name\":\"info\"}],\"createArchive\":false}\n"
It seems not to try to make an archive.

laalaguer · 2023-06-14T07:13:46Z

Report the same issue here.

Server:
v1.5.4 - binary

Client:
Extension 0.8.6 in Firefox and tested with unpacked zip in Chrome (plain environment, no other extensions).

Url:
https://book.douban.com/subject/35876121/

Error:
Using ./shiori add on the server creates a correct cache page and archive page on the server. But using an extension just creates an entry + URL. No cache and no archive page were created.

Possible help:
The database is upgraded as instructed in README from prior <1.5.0

laalaguer · 2023-06-14T07:24:37Z

Update:

I tried to down grade the server, using binary version 1.5.1 + 0.8.6 ext is correctly creating the bookmarks (although very slow).

vensko · 2023-06-23T18:57:45Z

It's an API issue, /api/bookmarks/ext is broken. /api/bookmarks works as intended. They use the same payload, so even simple update to use /api/bookmarks with added createArchive: true fixes the issue https://github.com/go-shiori/shiori-web-ext/blob/master/js/background-script.js#L195

crawfordlong added the type:bug label Mar 20, 2023

fmartingr transferred this issue from go-shiori/shiori Jul 6, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Shiori extension never creates archive #48

Shiori extension never creates archive #48

crawfordlong commented Mar 20, 2023

rmayergfx commented Apr 29, 2023

Alistair1231 commented May 21, 2023 •

edited

Loading

rmayergfx commented May 22, 2023

Alistair1231 commented May 25, 2023 •

edited

Loading

rmayergfx commented May 25, 2023

Alistair1231 commented May 25, 2023

laalaguer commented Jun 14, 2023 •

edited

Loading

laalaguer commented Jun 14, 2023

vensko commented Jun 23, 2023 •

edited

Loading

Shiori extension never creates archive #48

Shiori extension never creates archive #48

Comments

crawfordlong commented Mar 20, 2023

Data

Describe the bug / actual behavior

Expected behavior

To Reproduce

Notes

rmayergfx commented Apr 29, 2023

Alistair1231 commented May 21, 2023 • edited Loading

rmayergfx commented May 22, 2023

Alistair1231 commented May 25, 2023 • edited Loading

rmayergfx commented May 25, 2023

Alistair1231 commented May 25, 2023

laalaguer commented Jun 14, 2023 • edited Loading

laalaguer commented Jun 14, 2023

vensko commented Jun 23, 2023 • edited Loading

Alistair1231 commented May 21, 2023 •

edited

Loading

Alistair1231 commented May 25, 2023 •

edited

Loading

laalaguer commented Jun 14, 2023 •

edited

Loading

vensko commented Jun 23, 2023 •

edited

Loading