Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Lancache isn't working #138

Open
ramzess1 opened this issue Jan 10, 2019 · 46 comments
Open

Lancache isn't working #138

ramzess1 opened this issue Jan 10, 2019 · 46 comments

Comments

@ramzess1
Copy link

Hello,

I have installed the Lancache server. Lancache, Unbound and NGINX are activated. Forwarding addresses in Unbound file is working well.
But when i add an entry to test the cache for example :
local-zone: "download.microsoft.com." redirect
local-data: "download.microsoft.com. 600 IN A 1..."

Lancache is doing nothing, there is no logs and i got a request time out from the PC client.
The dns also isn't resolving the download.microsoft.com from my pc.

For the other forwarded sites the dns is resolving.

Thank you for your response.
@nexusofdoom
Copy link
Collaborator

I may be able to help tonight team-viewer ssh access to box?

@bntjah
Copy link
Owner

bntjah commented Jan 11, 2019 via email

@ramzess1
Copy link
Author

Problem solved, it was the ipv6 NIC enabled so i unchecked it.

I have another issue using this test link : http://download.microsoft.com/download/0/9/5/0953E553-3BB6-44B1-8973-106F1B7E5049/wmp11-windowsxp-x86-enu.exe

It gives me:
404 Not Found
nginx/1.13.4

This problem is lnked to the name download.microsoft.com added in the unbound file.

@ramzess1
Copy link
Author

capture

@ramzess1
Copy link
Author

capture

@bntjah
Copy link
Owner

bntjah commented Jan 11, 2019 via email

@ramzess1
Copy link
Author

Do you mean it should work for other sites? because i am only testing the "download.microsoft.com" on my windows PC

@bntjah
Copy link
Owner

bntjah commented Jan 11, 2019 via email

@ramzess1
Copy link
Author

There's no logs in:
access_log /srv/lancache/logs/Access/
access_log /srv/lancache/logs/Keys/
error_log /srv/lancache/logs/Errors/

And find next the microsoft host file and the hosts file.

And thank you for you help.

@ramzess1
Copy link
Author

microsoft

@ramzess1
Copy link
Author

hosts

@bntjah
Copy link
Owner

bntjah commented Jan 11, 2019 via email

@ramzess1
Copy link
Author

I got this log error in NGINX logs:
2019/01/11 12:22:23 [error] 7092#7092: *1 open() "/usr/local/nginx/html/download/0/9/5/0953E553-3BB6-44B1-8973-106F1B7E5049/wmp11-windowsxp-x86-enu.exe" failed (2: No such file or directory)

@bntjah
Copy link
Owner

bntjah commented Jan 11, 2019 via email

@ramzess1
Copy link
Author

capture

@ramzess1
Copy link
Author

capture

@ramzess1
Copy link
Author

capture

@ramzess1
Copy link
Author

capture

@ramzess1
Copy link
Author

I only changed the proxy bind variable in lancache-microsoft file

@bntjah
Copy link
Owner

bntjah commented Jan 11, 2019 via email

@ramzess1
Copy link
Author

The screenshots sent of nginx.conf are for the executable file given by nginx -t.

The other one has the same config as in Github.

The error seems to be a location error for the get http links.

@ramzess1
Copy link
Author

Everything is activated but the problem nginx isn't downloading the requested webpage.

The location inside the executor of nginx in the screenshot sent is showing "root: html" but in that folder there is only index.html the nginx default page.

I don't know if i should change the location inside the nginx executor file or there's anything else.

@nexusofdoom
Copy link
Collaborator

are you downloading as http or https ?

@nexusofdoom
Copy link
Collaborator

nexusofdoom commented Jan 11, 2019 •
edited
Loading

my setup is working
run this
sudo httpry -i ethX "X is what your network is setup as"
what do you see?

#example of what I see
2019-01-11 11:00:48 10.0.2.232 192.168.30.232 > GET download.microsoft.com /download/0/9/5/0953E553-3BB6-44B1-8973-106F1B7E5049/wmp11-windowsxp-x86-enu.exe HTTP/1.1 - -
2019-01-11 11:00:48 192.168.30.232 10.0.2.232 < - - - HTTP/1.1 200 OK

#my logs
/srv/lancache/logs/Keys/micrsoft.log
[11/Jan/2019:11:00:42 -0600] - REQUEST="microsoft/download/0/9/5/0953E553-3BB6-44B1-8973-106F1B7E5049/wmp11-windowsxp-x86-enu.exe" - KEY="microsoft/download/0/9/5/0953E553-3BB6-44B1-8973-106F1B7E5049/wmp11-windowsxp-x86-enu.exe -" - CACHE="HIT"
[11/Jan/2019:11:00:48 -0600] - REQUEST="microsoft/download/0/9/5/0953E553-3BB6-44B1-8973-106F1B7E5049/wmp11-windowsxp-x86-enu.exe" - KEY="microsoft/download/0/9/5/0953E553-3BB6-44B1-8973-106F1B7E5049/wmp11-windowsxp-x86-enu.exe -" - CACHE="HIT"

@ramzess1
Copy link
Author

Ok nexusofdoom i will try it on Monday.

It is an http link and was always showing 404 not found because it is pointing to etc/local/nginx/html/"the requested page file" which doesn't exist. I figured this by looking at nginx/logs/error.log as sent in one of the last comments.

This location only contains the default page index.html and the 505.. error file.

So i think nginx isn't downloading the file.

Https downloads doesn't work with lancache right? Because while downloading https from microsoft it dowloads the file normally but it doesn't cache it.

Thank you all for your replies.

@nexusofdoom
Copy link
Collaborator

If you have a spare pc box or vm I would try reloading linux and lancache and see how it works.

@raz3r83
Copy link
Contributor

raz3r83 commented Jan 12, 2019

what's the output of:
service lancache status

@ramzess1
Copy link
Author

Lancache is activated.

Nexusofdoom i will try to reinstall nginx, maybe there's something in the installation but on Monday because i don't have my pc now.

Can you send me the steps to install the full nginx please?

@raz3r83
Copy link
Contributor

raz3r83 commented Jan 12, 2019

Easiest way to install is using the install script https://github.com/nexusofdoom/lancache-installer
Written by nexusofdoom :)

Its been a while since I tried a manual install but you have cant just run apt-get install nginx as you have to load additional modules so have to run a compile, instructions are in the readme I believe.

@ramzess1
Copy link
Author

These are the steps that i made:

  1. Configure the network interface

2)apt-get install curl git unbound build-essential libpcre3 zlib1g-dev libreadline-dev libncurses5-dev libssl-dev httpry libudns0 libudns-dev libev4 libev-dev devscripts automake libtool autoconf autotools-dev cdbs debhelper dh-autoreconf dpkg-dev gettext pkg-config fakeroot libpcre3-dev libgd2-xpm-dev libgeoip-dev tcpdump -y

3)curl http://nginx.org/download/nginx-1.13.4.tar.gz | tar zx

cd nginx-1.13.4
curl "http://labs.frickle.com/files/ngx_cache_purge-2.3.tar.gz" | tar zx
git clone https://github.com/multiplay/nginx-range-cache/ $PWD/nginx-range-cache
curl "https://codeload.github.com/wandenberg/nginx-push-stream-module/tar.gz/0.5.1?dummy=/wandenberg-nginx-push-stream-module-0.5.1_GH0.tar.gz" | tar zx

4)git clone -b master http://github.com/bntjah/lancache

  1. cd nginx-1.13.4
    patch -p1 <$PWD/nginx-range-cache/range_filter.patch
    ./configure --modules-path=$PWD --with-cc-opt='-I /usr/local/include' --with-ld-opt='-L /usr/local/lib' --conf-path=/usr/local/nginx/nginx.conf --sbin-path=/usr/local/sbin/nginx --pid-path=/var/run/nginx.pid --with-file-aio --add-module=$PWD/ngx_cache_purge-2.3 --with-http_flv_module --with-http_geoip_module=dynamic --with-http_gzip_static_module --with-http_image_filter_module=dynamic --with-http_mp4_module --add-module=$PWD/nginx-range-cache --with-http_realip_module --with-http_slice_module --with-http_stub_status_module --with-pcre --with-http_v2_module --with-stream=dynamic --with-stream_ssl_module --with-stream_ssl_preread_module --with-http_ssl_module --add-module=$PWD/nginx-push-stream-module-0.5.1 --with-threads
    make
    make install

  2. Adding the virtual interfaces with ips in the same network as network interface's ip

7)adduser --system --no-create-home lancache
addgroup --system lancache
usermod -aG lancache lancache

8)mkdir -p /srv/lancache/data/{microsoft,installs,other,tmp,hirez,origin,riot,gog,sony,steam,wargaming,arenanetworks,uplay,glyph,zenimax,digitalextremes,pearlabyss, blizzard, apple}
mkdir -p /srv/lancache/logs/{Errors,Keys,Access}

9)chown -R lancache:lancache /srv/lancache

10)cp -R ~/lancache/conf /usr/local/nginx/

11)Replace the proxy-bind variable in each lancache-"service-name" file and in the hosts file

12)cp ~/lancache/init.d/lancache /etc/init.d/

13)chmod +x /etc/init.d/lancache

14)update-rc.d lancache defaults

15)cp ~/lancache/limits.conf /etc/security/

16)cp ~/lancache/hosts /etc/ and changed the virtual ip near each lancache service

17)echo "net.ipv6.conf.all.disable_ipv6=1" >/etc/sysctl.d/disable-ipv6.conf
sysctl -p /etc/sysctl.d/disable-ipv6.conf

  1. git clone https://github.com/dlundquist/sniproxy
    curl https://raw.githubusercontent.com/OpenSourceLAN/origin-docker/master/sniproxy/sniproxy.conf -o /etc/sniproxy.conf
    cd sniproxy
    ./autogen.sh && ./configure && make check && make install
    cp ~/lancache/init.d/sniproxy /etc/init.d/
    chmod +x /etc/init.d/sniproxy
    update-rc.d sniproxy defaults

  2. cp ~/lancache/unbound/unbound.conf /etc/unbound
    Added the virtual ips of each service
    added the ip of network interface

20)service lancache start

21)service unbound start

Is there anything left?

@ramzess1
Copy link
Author

Nexusofdoom this script "https://github.com/nexusofdoom/lancache-installer"installs automatically the full lancache service ??

@ramzess1
Copy link
Author

I saw it, greatt!

So i just have to change the variables of ips, dns.. in the files and the run the script?

Thankss

@raz3r83
Copy link
Contributor

raz3r83 commented Jan 12, 2019

there's really very little you would need to change in nexus's script though it needs a little bit of an update.
1 warning it makes pretty drastic changes to the netplan which might cause you issues if you are bonding network ports together.

  1. issue its using ifconfig to try and identify the interface name for later use which has been depreciated and isn't installed in a new install.
  2. issue its using route to identify the network gateway which doesn't seem to be installed as default either.
  3. another check for interface name is using ip link show but its searching for ens and my network device names eth so doesn't pickup a value.

The only real affect the 3 issues have are that I had to go and update the netplan yaml to set interface name and gateway, everything else appears to be working without having changed anything.

This was on a fresh ubuntu 18.04 container running on a proxmox host.

@nexusofdoom
Copy link
Collaborator

I know that it works with "Ubuntu 18.04.1 Server"
http://cdimage.ubuntu.com/releases/18.04.1/release/ubuntu-18.04.1-server-amd64.iso

Did you install 18.04 or 18.04.1?

@ramzess1
Copy link
Author

I executed the script of nexusofdomm and i got this error of unbound file

-- Unit unbound.service has finished shutting down.
Jan 14 17:25:14 LANCache systemd[1]: unbound.service: Start request repeated too quickly.
Jan 14 17:25:14 LANCache systemd[1]: unbound.service: Failed with result 'exit-code'.
Jan 14 17:25:14 LANCache systemd[1]: Failed to start Unbound DNS server.
-- Subject: Unit unbound.service has failed
-- Defined-By: systemd
-- Support: http://www.ubuntu.com/support

-- Unit unbound.service has failed.

-- The result is RESULT.
Jan 14 17:25:14 LANCache systemd[1]: unbound.service: Start request repeated too quickly.
Jan 14 17:25:14 LANCache systemd[1]: unbound.service: Failed with result 'exit-code'.
Jan 14 17:25:14 LANCache systemd[1]: Failed to start Unbound DNS server.

@nexusofdoom
Copy link
Collaborator

Are you running Ubuntu 18.04.1 Server?

@ramzess1
Copy link
Author

yess

@nexusofdoom
Copy link
Collaborator

just did a fresh install and unbound is working no issues.
image

@nexusofdoom
Copy link
Collaborator

image

@ramzess1
Copy link
Author

Did you do apt-get update and apt-get upgrade before the installation?

@nexusofdoom
Copy link
Collaborator

The script does the updates

Update packages

echo "Installing package updates..."
universeCheck=$(apt-cache policy |grep universe)
if [[ -z $universeCheck ]]; then
echo "Adding universe repository..."
apt-add-repository universe
else
apt -y update
fi
apt -y upgrade

@nexusofdoom
Copy link
Collaborator

I tried this on two other computers and one of the computers the network card was having issues replaced and the installer worked great.

@ramzess1
Copy link
Author

Trying to rerun the script on a new ubuntu 18.04.1, it gives

Installing package updates...
Adding universe repository...
'universe' distribution component is already enabled for all sources.
Reading package lists... Done
Building dependency tree
Reading state information... Done
Calculating upgrade... Done
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Reading package lists... Done
Building dependency tree
Reading state information... Done
Calculating upgrade... Done
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Installing required updates...
Reading package lists... Done
Building dependency tree

@nexusofdoom
Copy link
Collaborator

Sorry don't know what to say it updates and it also updates the kernel.
I tried again today and its working. fresh install of ubuntu

Download url for Ubuntu 18.04.1 Server
http://cdimage.ubuntu.com/releases/18.04.1/release/ubuntu-18.04.1-server-amd64.iso

You will need 18 avaliable IP's example 192.168.0.2 - 192.168.0.20 used for lancache
Clone the git repo
git clone -b master http://github.com/nexusofdoom/lancache-installer

cd lancache-installer

Run scripts with sudo
Run
sudo ./install-lancache.sh

@ramzess1
Copy link
Author

Configuration file '/etc/default/sniproxy'
==> File on system created by you or by a script.
==> File also in package provided by package maintainer.
What would you like to do about it ? Your options are:
Y or I : install the package maintainer's version
N or O : keep your currently-installed version
D : show the differences between the versions
Z : start a shell to examine the situation
The default action is to keep your current version.
*** sniproxy (Y/I/N/O/D/Z) [default=N] ?

which one please?

@ramzess1
Copy link
Author

Thank you all for your help, it works correctly.

Steps for new lancache installers :
-Ubuntu 18.04.1 with OpenSSH only
-apt -y update
-Run the script
-"Y" for any asked question
-Reboot at the end

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@bntjah @nexusofdoom @raz3r83 @ramzess1