-
Notifications
You must be signed in to change notification settings - Fork 258
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fanbox now uses Cloudflare #1342
Comments
I was just about to report this. I noticed this on my phone I think late last night where there was a captcha but wasn't sure why it was there then today discovered the downloader couldn't work because of it. I enabled http debugging and this might provide more insight. This request gives an indicator that a CloudFlare challenge was given to be handled. https://gist.github.com/shinji257/f0d3a4defa75037d1c61dd534a628750 (Reposted with new gist -- removed fanbox session id) |
Theoretically, this will probably just be temporary and resolve itself on it's own, going away after "whatever" it is that needed Cloudflare captchas is over. |
Luckily this only impacts Fanbox. Downloads from Pixiv still work fine. |
If this doesn't get removed, I have a local version now that is able to load the cf_clearance and __cf_bm cookies copied from a browser that has bypassed the check, which works as long as you mirror the user-agent of that browser in config.ini. I doubt those cookies would stay valid between download sessions for very long but at I was at least able to update rips of all my paid accounts through pixivutil once I copied them over. |
Thanks for this work around. I was also able to download my supported fanbox users by adding those two cookies to my config and then using them in the fanboxLoginUsingCookie function in my local copy. |
If you have a VPN you can set it for a Japanese IP and it will bypass Cloudflare captchas. I was able to download with no issues once I changed locations. |
How do you add the other cookies to the config? Right now there is only the cookieFanbox value that is the FANBOXSESSID. |
They're not currently supported/added by main branch, I had to add them in and also edit the cookie loading function to allow adding in a cookie value beyond the session id, as the current method resets the cookie jar on use and only sets that one cookie type. I just knew about these being a thing from my work with gallery-dl code so I knew that as long as you match those and the user-agent you can often bypass those captcha checks at least in the short term. |
how and where did you edit the cookie loading function? |
Is this still needed? I wasn't getting any Cloudflare when accessing from my location anymore. For reference, the way I did it for my local copy is noted below. Pretty hacky IMO, but it was working when I was still getting Cloudflare for Fanbox. This only adds the cookies for Fanbox, not Pixiv.
[Authentication]
cf_clearance = <cf_clearance COOKIE FROM BROWSER>
cf_bm = <__cf_bm COOKIE FROM BROWSER>
ConfigItem("Authentication", "cf_clearance", ""),
ConfigItem("Authentication", "cf_bm", ""),
if self._config.cf_clearance != "":
ck1 = http.cookiejar.Cookie(version=0, name='cf_clearance', value=self._config.cf_clearance, port=None,
port_specified=False, domain='fanbox.cc', domain_specified=False,
domain_initial_dot=False, path='/', path_specified=True,
secure=False, expires=None, discard=True, comment=None,
comment_url=None, rest={'HttpOnly': None}, rfc2109=False)
self.addCookie(ck1)
if self._config.cf_bm != "":
ck2 = http.cookiejar.Cookie(version=0, name='__cf_bm', value=self._config.cf_bm, port=None,
port_specified=False, domain='fanbox.cc', domain_specified=False,
domain_initial_dot=False, path='/', path_specified=True,
secure=False, expires=None, discard=True, comment=None,
comment_url=None, rest={'HttpOnly': None}, rfc2109=False)
self.addCookie(ck2) |
Yes. I'm still getting the cloudflare challenge here when using the default build. If I VPN to Japan the challenge is subverted. |
Thanks for this. I added this code to my local copy and it works now. |
Sry, I'm new to this in fanboxLoginUsingCookie |
I added it here: PixivUtil2/PixivBrowserFactory.py Line 384 in d0bfaae |
guess im doing something wrong cuz its not working for me :c |
The above instructions worked for me. Let's try not to flood the issue with overly large quotes. Someone could make a temporary fork with the code, unless @Nandaka decides to integrate this upstream somehow. |
@biggestsonicfan sure, just send me a pull request 😄 |
@biggestsonicfan I'm getting the same error as @usernameisalreadytakendammit after pulling the latest commit, I checked the cookies were correct and my useragent is the same as my browser |
@FriedGenera Edit: Also, make sure it's the Fanbox cookies, not the Pixiv cookies. |
@HetareKing I did what you say, still not working. |
Because its the last day of the month and time was running out but i still couldnt get that script to work, as a last ditch effort, i used urban vpn to get a japanese ip for free. That was the only way for me to get it to work :c |
EDIT: So the error appears to be happening here: PixivUtil2/PixivBrowserFactory.py Lines 204 to 215 in f09af10
EDIT2: Ah, of course, we're getting a EDIT3: Graceful handling of the Cloudflare CAPTCHA challenge failures should be handled in #1345. |
People use Now, keeping it as-is, and added cf_clearance and cf_bm cookies in config, I still can't download fanbox just like @Zombie10 |
Using @biggestsonicfan's PR: cf_clearance and cf_bm both set to Edit: Changed the UA to my browser's full UA, Fanbox now works again. Thanks! (Might want to update the version shown by the way, it's still using Edit2: Beam resets every hour which triggers the captcha again and both cookie values have to be set again manually or the user will run into the above error. |
I cannot find the cf_clearance cookie at all. Where is it supposed to be found? EDIT: Found it using the developer tools. Doesn't work, though. And changing my UA results in an immediate failure to login at all. |
Can you please write occasionally in a way that non-experts can understand? Thanks |
How did you avoid having the Pixiv login failure that would result from using the full UA? |
Can you move your logs to a different folder (or clear them if you're okay with that) and send me a fresh log of just trying to download one fanbox item? You can remove your id or anything else in the log you don't want others to see but I'm curious if there's anything else in there that will help me figure this out. I cannot reproduce this. |
Here you go. |
I think I found the problem. Lol. We are all using the Python interpreter to run Executing |
Wait, what??? Then what am I supposed to execute, then?! |
Hey there! Remember when I told you...
I was not joking. You are supposed to be executing PixivUtil2.py directly. As an example, on Windows you could use WinPython to give yourself the appropriate libraries and binaries, and then open PixivUtil2.py using python.exe. |
I got this error when I tried to execute directly from the command prompt UI.
What do I do now? |
You should not be cluttering this issue with basic Python troubleshooting questions that are wholly unrelated to PixivUtil2 and the recent Cloudflare transition. However, I will be incredibly nice and hold your hand just this once. As indicated by this answer on stackoverflow, you can try to update requests-toolbelt in order to get a version that is compatible with urllib3 2.0.0. You would do this by running the command Considering your inexperience, however, this will likely just result in more errors. For example, did you read the part on the main github readme about installing all of PixivUtil2's dependencies listed in requirements.txt? If not, make sure you install all of those dependencies as well. Best of luck. And if all this is over your head, just wait until a new version is released. |
It's finally working! Thank you for being patient with me, nomakewan. Those updates were the last pieces missing from the puzzle. |
The latest version is from January 2023 and there will probably be no new updates. And I don't know enough about this shit. Greetings to the developer: I'm waiting! |
Last update is over a year old, any hope?? |
The issue is no longer present on my end. Seems like Fanbox has downgraded their Cloudflare security mode? |
@gnarf1975 https://github.com/Nandaka/PixivUtil2/releases/tag/v20240703 also backup your old files! Got some issue with compilation to exe file, so I still recommend to run from source code... |
seems it only works on firefox for me. for Google chrome i still get the error cuz it doesnt accept google chrome user agent x) |
Pulled down the newest version. As per this issue I logged into Pixiv and Fanbox using Firefox then grabbed both of their session cookies. Updated user agent to match the Firefox browser used. Used CF tokens from Fanbox (they have different tokens). Working like a charm without a VPN. Thanks! |
Newest release. UserAgent updated. cookieFanbox and cf_clearance accounted for. Still doesn't work. "Error processing FANBOX post: ==> (post id) Failed FANBOX Cloudflare CAPTCHA challenge, please check your cookie and user-agent settings." Works with VPN but I thought the issue was resolved so as to not need that. Readme does not help at all. |
You are missing the The Readme can not help you as this issue is too new and ongoing to document. If the issue persists, perhaps the user/pass functionality of PixivUtil2 should be removed altogether in favor of parsing cookies from the browser. |
Is there an explanation somewhere for how to capture these right now? cf_clearance and cf_bm? |
use inspect elements |
useragent = Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/115.0 |
If that's the useragent of your firefox browser then yes |
thats in my config file |
Is that the one thats in the config file by default? |
@usernameisalreadytakendammit is this on a completely new profile on Firefox? I keep getting captcha error |
@biggestsonicfan @usernameisalreadytakendammit Yea seems that on a completely freshly built chromium browser it works just fine, I'll check which of the settings in the Firefox menu/about:config is causing the issues with the Cloudflare cookies |
pixiv fanbox removed cf_clearance ? |
Prerequisites
Description
All fanbox functions now create a
AttributeError: 'str' object has no attribute 'decode'
error due to new protections.Steps to Reproduce
Expected behavior:
Fanbox downloads
Actual behavior:
Fanbox blocked
Log file:
pixivutil.log
Versions
Latest git as of d0bfaae
You can get this information from executing
PixivUtil2.py --help
.Latest version available in https://github.com/Nandaka/PixivUtil2/releases
The text was updated successfully, but these errors were encountered: