Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG] interface conversion: interface {} is nil, not int64 #523

Open
Cyberes opened this issue Oct 2, 2023 · 10 comments · May be fixed by #528
Open

[BUG] interface conversion: interface {} is nil, not int64 #523

Cyberes opened this issue Oct 2, 2023 · 10 comments · May be fixed by #528
Labels
bug Something isn't working

Comments

@Cyberes
Copy link

Cyberes commented Oct 2, 2023 •
edited
Loading

After about a month of collecting data it becomes impossible to load the metrics due to a 500 internal server error.

This has been a long-running issue and affects on Docker as well. I also tried building from source using commit 4b1d9dc2d3f5388440a6746c3bdce2b8e2bee91e. The only way to fix this is to nuke the InfluxDB database.

Metrics seem to be collected fine (as seen in the "last updated" field next to each disk), so the issue must be with displaying them as an error is thrown when I click a drive to view its metrics.

Log Files

Oct 02 16:04:21 scrutiny scrutiny-web[1318491]: 2023/10/02 16:04:21 [Recovery] 2023/10/02 - 16:04:21 panic recovered:
Oct 02 16:04:21 scrutiny scrutiny-web[1318491]: interface conversion: interface {} is nil, not int64
Oct 02 16:04:21 scrutiny scrutiny-web[1318491]: /usr/local/go/src/runtime/iface.go:263 (0x40dba4)
Oct 02 16:04:21 scrutiny scrutiny-web[1318491]:         panicdottypeE: panic(&TypeAssertionError{iface, have, want, ""})
Oct 02 16:04:21 scrutiny scrutiny-web[1318491]: /home/username/scrutiny/webapp/backend/pkg/models/measurements/smart.go:69 (0x83b6ed)
Oct 02 16:04:21 scrutiny scrutiny-web[1318491]: /home/username/scrutiny/webapp/backend/pkg/database/scrutiny_repository_device_smart_attributes.go:54 (0xc22ebb)
Oct 02 16:04:21 scrutiny scrutiny-web[1318491]: /home/username/scrutiny/webapp/backend/pkg/web/handler/get_device_details.go:27 (0xdbdeca)
Oct 02 16:04:21 scrutiny scrutiny-web[1318491]: /home/username/scrutiny/vendor/github.com/gin-gonic/gin/context.go:161 (0xd821d3)
Oct 02 16:04:21 scrutiny scrutiny-web[1318491]: /home/username/scrutiny/vendor/github.com/gin-gonic/gin/recovery.go:83 (0xd821c1)
Oct 02 16:04:21 scrutiny scrutiny-web[1318491]: /home/username/scrutiny/vendor/github.com/gin-gonic/gin/context.go:161 (0xd77f0a)
Oct 02 16:04:21 scrutiny scrutiny-web[1318491]: /home/username/scrutiny/webapp/backend/pkg/web/middleware/config.go:11 (0xdc4944)
Oct 02 16:04:21 scrutiny scrutiny-web[1318491]: /home/username/scrutiny/vendor/github.com/gin-gonic/gin/context.go:161 (0xdc33b9)
Oct 02 16:04:21 scrutiny scrutiny-web[1318491]: /home/username/scrutiny/webapp/backend/pkg/web/middleware/repository.go:29 (0xdc33a7)
Oct 02 16:04:21 scrutiny scrutiny-web[1318491]: /home/username/scrutiny/vendor/github.com/gin-gonic/gin/context.go:161 (0xdc23b4)
Oct 02 16:04:21 scrutiny scrutiny-web[1318491]: /home/username/scrutiny/webapp/backend/pkg/web/middleware/logger.go:56 (0xdc2399)
Oct 02 16:04:21 scrutiny scrutiny-web[1318491]: /home/username/scrutiny/vendor/github.com/gin-gonic/gin/context.go:161 (0xd81208)
Oct 02 16:04:21 scrutiny scrutiny-web[1318491]: /home/username/scrutiny/vendor/github.com/gin-gonic/gin/gin.go:409 (0xd80e62)
Oct 02 16:04:21 scrutiny scrutiny-web[1318491]: /home/username/scrutiny/vendor/github.com/gin-gonic/gin/gin.go:367 (0xd809b3)
Oct 02 16:04:21 scrutiny scrutiny-web[1318491]: /usr/local/go/src/net/http/server.go:2938 (0x77b6ad)
Oct 02 16:04:21 scrutiny scrutiny-web[1318491]:         serverHandler.ServeHTTP: handler.ServeHTTP(rw, req)
Oct 02 16:04:21 scrutiny scrutiny-web[1318491]: /usr/local/go/src/net/http/server.go:2009 (0x777593)
Oct 02 16:04:21 scrutiny scrutiny-web[1318491]:         (*conn).serve: serverHandler{c.server}.ServeHTTP(w, w.req)
Oct 02 16:04:21 scrutiny scrutiny-web[1318491]: /usr/local/go/src/runtime/asm_amd64.s:1650 (0x46b0a0)
Oct 02 16:04:21 scrutiny scrutiny-web[1318491]:         goexit: BYTE        $0x90        // NOP

InfluxDB is created with:

docker run -d --restart=always -p 8086:8086 \
  --network="host" \
  -v /var/docker-data/influxdb2:/var/lib/influxdb2 \
  --name influxdb \
  influxdb:2.2

I run the web dashboard in an LXC container which all my hosts send their data to.
@Cyberes Cyberes added the bug Something isn't working label Oct 2, 2023
@Cyberes
Copy link
Author

Cyberes commented Oct 15, 2023 •
edited
Loading

@AnalogJ I know you're busy with Fasten, but can you take a look at this issue? It's been a recurring problem and is probably a bug due to the traceback.

@AnalogJ AnalogJ linked a pull request Oct 17, 2023 that will close this issue
Open
@AnalogJ
Copy link
Owner

AnalogJ commented Oct 17, 2023

hey @Cyberes I think I fixed this in the beta branch. Can you test the beta-omnibus docker image?

@Cyberes
Copy link
Author

Cyberes commented Oct 18, 2023 •
edited
Loading

Great! I compiled the web binary from the beta branch but got this when loading /web/:

Oct 17 22:14:04 scrutiny scrutiny-web[1904255]: time="2023-10-17T22:14:04-06:00" level=info msg="172.0.1.1 - scrutiny [17/Oct/2023:22:14:04 -0600] \"GET /web/\" 200 7467 \"\" \"Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/117.0\" (1ms)" clientIP=xxx.xxx.xxx.xxx hostname=scrutiny latency=1 method=GET path=/web/ referer= respLength=7467 statusCode=200 type=web userAgent="Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/117.0"
Oct 17 22:14:07 scrutiny scrutiny-web[1904255]: 2023/10/17 22:14:07 [Recovery] 2023/10/17 - 22:14:07 panic recovered:
Oct 17 22:14:07 scrutiny scrutiny-web[1904255]: runtime error: invalid memory address or nil pointer dereference
Oct 17 22:14:07 scrutiny scrutiny-web[1904255]: /usr/local/go/src/runtime/panic.go:261 (0x44f0b7)
Oct 17 22:14:07 scrutiny scrutiny-web[1904255]:         panicmem: panic(memoryError)
Oct 17 22:14:07 scrutiny scrutiny-web[1904255]: /usr/local/go/src/runtime/signal_unix.go:861 (0x44f085)
Oct 17 22:14:07 scrutiny scrutiny-web[1904255]:         sigpanic: panicmem()
Oct 17 22:14:07 scrutiny scrutiny-web[1904255]: /home/username/scrutiny/webapp/backend/pkg/database/scrutiny_repository.go:436 (0xc213fe)
Oct 17 22:14:07 scrutiny scrutiny-web[1904255]: /home/username/scrutiny/webapp/backend/pkg/web/handler/get_devices_summary.go:14 (0xdbe8de)
Oct 17 22:14:07 scrutiny scrutiny-web[1904255]: /home/username/scrutiny/vendor/github.com/gin-gonic/gin/context.go:161 (0xd82653)
Oct 17 22:14:07 scrutiny scrutiny-web[1904255]: /home/username/scrutiny/vendor/github.com/gin-gonic/gin/recovery.go:83 (0xd82641)
Oct 17 22:14:07 scrutiny scrutiny-web[1904255]: /home/username/scrutiny/vendor/github.com/gin-gonic/gin/context.go:161 (0xd7838a)
Oct 17 22:14:07 scrutiny scrutiny-web[1904255]: /home/username/scrutiny/webapp/backend/pkg/web/middleware/config.go:11 (0xdc4dc4)
Oct 17 22:14:07 scrutiny scrutiny-web[1904255]: /home/username/scrutiny/vendor/github.com/gin-gonic/gin/context.go:161 (0xdc3839)
Oct 17 22:14:07 scrutiny scrutiny-web[1904255]: /home/username/scrutiny/webapp/backend/pkg/web/middleware/repository.go:29 (0xdc3827)
Oct 17 22:14:07 scrutiny scrutiny-web[1904255]: /home/username/scrutiny/vendor/github.com/gin-gonic/gin/context.go:161 (0xdc2834)
Oct 17 22:14:07 scrutiny scrutiny-web[1904255]: /home/username/scrutiny/webapp/backend/pkg/web/middleware/logger.go:56 (0xdc2819)
Oct 17 22:14:07 scrutiny scrutiny-web[1904255]: /home/username/scrutiny/vendor/github.com/gin-gonic/gin/context.go:161 (0xd81688)
Oct 17 22:14:07 scrutiny scrutiny-web[1904255]: /home/username/scrutiny/vendor/github.com/gin-gonic/gin/gin.go:409 (0xd812e2)
Oct 17 22:14:07 scrutiny scrutiny-web[1904255]: /home/username/scrutiny/vendor/github.com/gin-gonic/gin/gin.go:367 (0xd80e33)
Oct 17 22:14:07 scrutiny scrutiny-web[1904255]: /usr/local/go/src/net/http/server.go:2938 (0x77b6ad)
Oct 17 22:14:07 scrutiny scrutiny-web[1904255]:         serverHandler.ServeHTTP: handler.ServeHTTP(rw, req)
Oct 17 22:14:07 scrutiny scrutiny-web[1904255]: /usr/local/go/src/net/http/server.go:2009 (0x777593)
Oct 17 22:14:07 scrutiny scrutiny-web[1904255]:         (*conn).serve: serverHandler{c.server}.ServeHTTP(w, w.req)
Oct 17 22:14:07 scrutiny scrutiny-web[1904255]: /usr/local/go/src/runtime/asm_amd64.s:1650 (0x46b0a0)
Oct 17 22:14:07 scrutiny scrutiny-web[1904255]:         goexit: BYTE        $0x90        // NOP
Oct 17 22:14:07 scrutiny scrutiny-web[1904255]:

If you really want me to try it with Docker, let me know how I can use the beta scrutiny:master-web

@Cyberes
Copy link
Author

Cyberes commented Nov 6, 2023

@AnalogJ I'm currently in a situation where I need to view my hard drive metrics so I'd have to reset the InfluxDB database. I'm open to other solutions if there's another patch or workaround you'd like me to try.

@ovizii
Copy link

ovizii commented Feb 16, 2024

I'm getting this error:



scrutiny  |
scrutiny  |
scrutiny  | 2024/02/16 22:26:06 [Recovery] 2024/02/16 - 22:26:06 panic recovered:
scrutiny  | runtime error: invalid memory address or nil pointer dereference
scrutiny  | /usr/local/go/src/runtime/panic.go:260 (0x44d01c)
scrutiny  | /usr/local/go/src/runtime/signal_unix.go:841 (0x44cfec)
scrutiny  | /go/src/github.com/analogj/scrutiny/webapp/backend/pkg/database/scrutiny_repository.go:436 (0xc61e34)
scrutiny  | /go/src/github.com/analogj/scrutiny/webapp/backend/pkg/web/handler/get_devices_summary.go:14 (0xe0b988)
scrutiny  | /go/src/github.com/analogj/scrutiny/vendor/github.com/gin-gonic/gin/context.go:161 (0xdcd0da)
scrutiny  | /go/src/github.com/analogj/scrutiny/vendor/github.com/gin-gonic/gin/recovery.go:83 (0xdcd0c6)
scrutiny  | /go/src/github.com/analogj/scrutiny/vendor/github.com/gin-gonic/gin/context.go:161 (0xe0f6be)
scrutiny  | /go/src/github.com/analogj/scrutiny/webapp/backend/pkg/web/middleware/config.go:11 (0xe0f6a5)
scrutiny  | /go/src/github.com/analogj/scrutiny/vendor/github.com/gin-gonic/gin/context.go:161 (0xe10c1e)
scrutiny  | /go/src/github.com/analogj/scrutiny/webapp/backend/pkg/web/middleware/repository.go:29 (0xe10c05)
scrutiny  | /go/src/github.com/analogj/scrutiny/vendor/github.com/gin-gonic/gin/context.go:161 (0xe0fc13)
scrutiny  | /go/src/github.com/analogj/scrutiny/webapp/backend/pkg/web/middleware/logger.go:56 (0xe0fbee)
scrutiny  | /go/src/github.com/analogj/scrutiny/vendor/github.com/gin-gonic/gin/context.go:161 (0xdcc029)
scrutiny  | /go/src/github.com/analogj/scrutiny/vendor/github.com/gin-gonic/gin/gin.go:409 (0xdcbc77)
scrutiny  | /go/src/github.com/analogj/scrutiny/vendor/github.com/gin-gonic/gin/gin.go:367 (0xdcb793)
scrutiny  | /usr/local/go/src/net/http/server.go:2936 (0x7969d5)
scrutiny  | /usr/local/go/src/net/http/server.go:1995 (0x791ef1)
scrutiny  | /usr/local/go/src/runtime/asm_amd64.s:1598 (0x469280)
scrutiny  |

with:

services:
  scrutiny:
    image: ghcr.io/analogj/scrutiny:master-omnibus

Its been running for quite some time just fine, actually, I managed to access the web interface of scrutiny, delete an old HD, got asked if I was sure as I was going to lose historical data and then everything hang with this error. Restarting the container does no good.

@Cyberes
Copy link
Author

Cyberes commented Feb 16, 2024

Sounds like my issue.

This was referenced Feb 23, 2024
Closed
Open
@AnalogJ
Copy link
Owner

AnalogJ commented Mar 18, 2024

we need copies of the smartctl json output to run against our tests and determine which key is failing to de-serialize.

@Cyberes
Copy link
Author

Cyberes commented Mar 20, 2024 •
edited
Loading

I re-created my database and it's been working for a couple months without any issues, so I don't have anything to give you right now. But if the error crops back up I'll certainly share the JSON.

As a side note, I replaced a few disks on a couple of my servers and I may have replaced the disk that was returning the bad JSON.

@ovizii
Copy link

ovizii commented Mar 20, 2024

I can make a copy of my scrutiny data by the weekend and then try to trigger the problem again by deleting an old HD from scrutiny's GUI. I'm just not sure where to find the "smartctl json output” you mentioned.

@ovizii
Copy link

ovizii commented Mar 20, 2024

OK, I made a copy of the InfluxDB and my scrutiny.db, went to the scrutiny GUI, deleted a device, refreshed and the GUI hangs.

I copied the docker logs and have removed my email and pushbullet secret and am happy to share it. Its quite long, I'll have to see if I can attach it here.

I can see the deletion in the logs, followed by a few other things happening in the background and then:

scrutiny  | 2024/03/20 16:11:45 [Recovery] 2024/03/20 - 16:11:45 panic recovered:
scrutiny  | runtime error: invalid memory address or nil pointer dereference
scrutiny  | /usr/local/go/src/runtime/panic.go:260 (0x44cffc)
scrutiny  | /usr/local/go/src/runtime/signal_unix.go:841 (0x44cfcc)
scrutiny  | /go/src/github.com/analogj/scrutiny/webapp/backend/pkg/database/scrutiny_repository.go:436 (0xc615d4)
scrutiny  | /go/src/github.com/analogj/scrutiny/webapp/backend/pkg/web/handler/get_devices_summary.go:14 (0xe0be68)
scrutiny  | /go/src/github.com/analogj/scrutiny/vendor/github.com/gin-gonic/gin/context.go:161 (0xdcd07a)
scrutiny  | /go/src/github.com/analogj/scrutiny/vendor/github.com/gin-gonic/gin/recovery.go:83 (0xdcd066)
scrutiny  | /go/src/github.com/analogj/scrutiny/vendor/github.com/gin-gonic/gin/context.go:161 (0xe0fbfe)
scrutiny  | /go/src/github.com/analogj/scrutiny/webapp/backend/pkg/web/middleware/config.go:11 (0xe0fbe5)
scrutiny  | /go/src/github.com/analogj/scrutiny/vendor/github.com/gin-gonic/gin/context.go:161 (0xe1115e)
scrutiny  | /go/src/github.com/analogj/scrutiny/webapp/backend/pkg/web/middleware/repository.go:29 (0xe11145)
scrutiny  | /go/src/github.com/analogj/scrutiny/vendor/github.com/gin-gonic/gin/context.go:161 (0xe10153)
scrutiny  | /go/src/github.com/analogj/scrutiny/webapp/backend/pkg/web/middleware/logger.go:56 (0xe1012e)
scrutiny  | /go/src/github.com/analogj/scrutiny/vendor/github.com/gin-gonic/gin/context.go:161 (0xdcbfc9)
scrutiny  | /go/src/github.com/analogj/scrutiny/vendor/github.com/gin-gonic/gin/gin.go:409 (0xdcbc17)
scrutiny  | /go/src/github.com/analogj/scrutiny/vendor/github.com/gin-gonic/gin/gin.go:367 (0xdcb733)
scrutiny  | /usr/local/go/src/net/http/server.go:2936 (0x7961d5)
scrutiny  | /usr/local/go/src/net/http/server.go:1995 (0x7916f1)
scrutiny  | /usr/local/go/src/runtime/asm_amd64.s:1598 (0x469260)

logs.log

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

safer attribute casting. AnalogJ/scrutiny
3 participants
@ovizii @AnalogJ @Cyberes