家庭组网-软路由

发表于 更新于 分类于 阅读次数：

最近在折腾家庭组网，买了一个软路由：N5105+16G+256 SSD。pve+ikuai+openwrt。由于有两条宽带（移动+电信），因此，使用ikuai进行分流，op进行科学。

测速

背景

之前在上个地方用的是电信+ESXI+iKuai+OpenWrt，由于电信给了公网IPV4，所以这套配置一直用的也很舒服。在现在这个地方已经装了移动的宽带，移动不给公网IPV4，通过使用ipv6的方式解决了公网访问，但是ipv6的情况ikuai没有防火墙配置，导致需要走op进行ipv6分配。那时候弄了半天，将ESXI改成PVE（其实是配置OpenWrt的防火墙时候，把策略都搞乱了。所以，重新搭建），成功使用ipv6（防火墙）。但存在不完美的地方，网卡需要经常重启，且openclash打开的情况，经常会出现访问国内网络不稳定（关了openclash也会出现）。因此，重新拉了一条电信宽带。

系统安装

PVE

下载对应iso，使用BalenaEtcher烧录到U盘。从U盘启动进行安装即可。开启直通。

1
2
3
4
5
6
7
8
9
10
11
12
13
nano /etc/default/grub
  GRUB_DEFAULT=0
  GRUB_TIMEOUT=5
  GRUB_DISTRIBUTOR=`lsb_release -i -s 2> /dev/null || echo Debian`
  GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on"
  GRUB_CMDLINE_LINUX=""
update-grub
nano /etc/modules
	vfio
  vfio_iommu_type1
  vfio_pci
  vfio_virqfd
update-initramfs -u -k all.

iKuai

创建虚拟机之后，加载iso之后配置对应LAN地址即可。需要添加PCIE设备，将所有网卡都添加进去。

PCI

OenWrt

创建虚拟机之后，删除硬盘之后将对应img添加即可。

1
qm importdisk 101 /var/lib/vz/template/iso/OpenWrt.img local-lvm

网络配置

iKuai

直接划分两个WAN口，分别用作电信、移动拨号。

ikuai

PPOE

将对应密码填入即可，将电信选择为默认线路。

DHCP

将网关地址设置为OpenWrt的地址、首选DNS也配置为OpenWrt的地址。

DNS

DNS也将首选DNS配置为OpenWrt地址即可。

跨三层应用

由于是旁路由模式，所以配置SNMP服务器IP为OpenWrt即可。

DDNS

由于电信给了公网IP，配置ddns便于外网访问。【高级应用】-【动态域名】里进行配置，由于使用的是dnspod.cn，在使用前需要申请对应TokenID、Token Key，并且需要现在对应的域名中先创建一条对应的A记录。

动态域名

配置完之后，在【网络配置】-【端口映射】里配置对应规则即可。

分流

暂未配置。（没想清楚如何分流更加合适，之后再配置）

OpenWrt

OpenWrt作为网关，并进行科学上网。

PVE

显卡直通

升级pve版本至7.4-3之后，修改以下配置。

换源

1
2
3
4
5
6
7
8
9
10
11
12
cat /dev/null > /etc/apt/sources.list

cat > /etc/apt/sources.list <<EOF
deb https://mirrors.tuna.tsinghua.edu.cn/debian/ bullseye main contrib non-free
deb-src https://mirrors.tuna.tsinghua.edu.cn/debian/ bullseye main contrib non-free
deb https://mirrors.tuna.tsinghua.edu.cn/debian/ bullseye-updates main contrib non-free
deb-src https://mirrors.tuna.tsinghua.edu.cn/debian/ bullseye-updates main contrib non-free
deb https://mirrors.tuna.tsinghua.edu.cn/debian/ bullseye-backports main contrib non-free
deb-src https://mirrors.tuna.tsinghua.edu.cn/debian/ bullseye-backports main contrib non-free
deb https://mirrors.tuna.tsinghua.edu.cn/debian-security bullseye-security main contrib non-free
deb-src https://mirrors.tuna.tsinghua.edu.cn/debian-security bullseye-security main contrib non-free
EOF

升级pre

1
apt update && apt install pve-kernel-5.19

下载依赖

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
mkdir -p /lib/firmware/i915 && cd /lib/firmware/i915
curl -LO https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/plain/i915/ehl_guc_70.1.1.bin
curl -LO https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/plain/i915/ehl_huc_9.0.0.bin
curl -LO https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/plain/i915/icl_dmc_ver1_09.bin

apt install software-properties-common

wget -qO – https://repositories.intel.com/graphics/intel-graphics.key |
apt-key add –
apt-add-repository 'deb [arch=amd64] https://repositories.intel.com/graphics/ubuntu focal main'

apt install intel-microcode -y
apt install intel-opencl-icd intel-level-zero-gpu level-zero intel-media-va-driver-non-free libmfx1

ls -l /dev/dri


安装lxc，并修改对应lxc配置

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
vim /etc/pve/lxc/102.conf # 根据lxc对应ID进行修改
  lxc.cgroup2.devices.allow: c 226:0 rwm
  lxc.cgroup2.devices.allow: c 226:128 rwm
  lxc.mount.entry: /dev/dri/card0 dev/dri/card0 none bind,optional,create=file
  lxc.mount.entry: /dev/dri/renderD128 dev/dri/renderD128 none bind,optional,create=file
  lxc.apparmor.profile: unconfined
  lxc.cap.drop:

vim /etc/modprobe.d/i915.conf
	options i915 enable_guc=3
	
vim /lib/systemd/system/rc-local.service
  [Install]
  WantedBy=multi-user.target

cat <<EOF >/etc/rc.local
#!/bin/sh -e
#
#rc.local
#
# This script is executed at the end of each multiuser runlevel.
# Make sure that the script will "exit 0" on success or any other
# value on error.
#
# In order to enable or disable this script just change the execution
# bits
#
# By default this script does nothing.**
chmod 777 /dev/dri/*
exit 0
EOF

chmod +x /etc/rc.local
systemctl enable rc-local.service

reboot

journalctl -b -o short-monotonic -k | egrep -i "i915|dmr|dmc|guc|huc"

配置完之后，安装embyServer，并通过NFS挂载磁盘（lxc需开启特权，并且开放NFS功能）

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
dpkg -i emby-server-deb_XXX_amd64.deb

systemctl enable emby-server
systemctl start emby-server

apt-get update && apt install intel-gpu-tools && sudo apt install nfs-common
install_gpu_top

cat <<EOF >/etc/rc.local
#!/bin/sh -e
#
#rc.local
#
# This script is executed at the end of each multiuser runlevel.
# Make sure that the script will "exit 0" on success or any other
# value on error.
#
# In order to enable or disable this script just change the execution
# bits
#
# By default this script does nothing.**
mount -t nfs ip:/volumnX/xxx /mnt/xxx
# mount -t nfs ip:/volumnX/xxx /mnt/xxx
# mount -t nfs ip:/volumnX/xxx /mnt/xxx
# mount -t nfs ip:/volumnX/xxx /mnt/xxx
# mount -t nfs ip:/volumnX/xxx /mnt/xxx
EOF

chmod +x /etc/rc.local
systemctl enable rc-local.service

温度监控

安装对应依赖

1
2
3
apt-get install lm-sensors
apt-get install hddtemp
apt-get install lm-sensors patch

修改/usr/share/perl5/PVE/API2/Nodes.pm文件，在ksm后面加入对应内容。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
$res->{ksm} = {
    shared => $meminfo->{memshared},
       };

# $res->{temperatures} = `sensors`;
$res->{sensors_json} = `sensors -j`; # JSON格式输出传感器数据
$res->{smartctl_nvme_json} = `smartctl -a -j /dev/nvme?`; # 读取 nvme 硬盘 S.M.A.R.T. 值，获取硬盘寿命、容量、温度等
$res->{cpusensors} = `lscpu | grep MHz`; # 读取 CPU 频率
# $res->{hddtemp} = `hddtemp /dav/sd?`; # JSON格式输出传感器数据

$res->{swap} = {
    free => $meminfo->{swapfree},
    total => $meminfo->{swaptotal},
    used => $meminfo->{swapused},
};

$res->{pveversion} = PVE::pvecfg::package() . "/" .
    PVE::pvecfg::version_text();

my $dinfo = df('/', 1);     # output is bytes

$res->{rootfs} = {
    total => $dinfo->{blocks},
    avail => $dinfo->{bavail},
    used => $dinfo->{used},
    free => $dinfo->{blocks} - $dinfo->{used},
};

return $res;
   }});

修改/usr/share/pve-manager/js/pvemanagerlib.js，在cpuinfo后面添加对应内容。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
{
    itemId: 'cpus',
    colspan: 2,
    printBar: false,
    title: gettext('CPU(s)'),
    textField: 'cpuinfo',
    renderer: Proxmox.Utils.render_cpu_model,
    value: '',
},
{
    itemId: 'sensinfo',
    colspan: 2,
    printBar: false,
    title: gettext('温度传感器'),  // WEB显示内容
    textField: 'sensors_json',
    renderer:function(value){
        value = JSON.parse(value.replaceAll('Â', ''));
        const c9 = value['coretemp-isa-0000']['Package id 0']['temp1_input'].toFixed(1);
        const c0 = value['coretemp-isa-0000']['Core 0']['temp2_input'].toFixed(1);
        const c1 = value['coretemp-isa-0000']['Core 1']['temp3_input'].toFixed(1);
        const c2 = value['coretemp-isa-0000']['Core 2']['temp4_input'].toFixed(1);
        const c3 = value['coretemp-isa-0000']['Core 3']['temp5_input'].toFixed(1);
        // const f1 = value['it8786-isa-0a40']['fan1']['fan1_input'].toFixed(1);
        const n1 = value['nvme-pci-0100']['Composite']['temp1_input'].toFixed(1);
        const a0 = value['acpitz-acpi-0']['temp1']['temp1_input'].toFixed(1);
        return `主板温度: ${a0}°C || CPU温度: ${c9}°C || NVME温度: ${n1}°C <br> CPU核心温度: ${c0}°C || ${c1}°C || ${c2}°C || ${c3}°C<br>`;  // 输出格式
    }
},
{	
    itemId: 'nvme_ssd',
    colspan: 2,
    printBar: false,
    title: gettext('NVME'),
    textField: 'smartctl_nvme_json',
    renderer: function(value) {
        value = JSON.parse(value);
        if (value['model_name']) {
            try {var model_name = value['model_name'];} catch(e) {var model_name = '';} 
            try {var percentage_used = ' | 使用寿命: ' + value['nvme_smart_health_information_log']['percentage_used'].toFixed(0) + '% ';} catch(e) {var percentage_used = '';} 
            try {var data_units_read = value['nvme_smart_health_information_log']['data_units_read']*512/1024/1024/1024;var data_units_read = '(读: ' + data_units_read.toFixed(2) + 'TB, ';} catch(e) {var data_units_read = '';} 
            try {var data_units_written = value['nvme_smart_health_information_log']['data_units_written']*512/1024/1024/1024;var data_units_written = '写: ' + data_units_written.toFixed(2) + 'TB)';} catch(e) {var data_units_written = '';} 
            try {var power_on_time = ' | 通电: ' + value['power_on_time']['hours'].toFixed(0) + '小时';} catch(e) {var power_on_time = '';} 
            try {var temperature = ' | 温度: ' + value['temperature']['current'].toFixed(1) + '°C';} catch(e) {var temperature = '';} 
            return `${model_name}${percentage_used}${data_units_read}${data_units_written}${power_on_time}${temperature}`;
        } else { 
            return `提示: 未安装硬盘或已直通硬盘控制器`;
        }
    }
},
{
    itemId: 'MHz',
    colspan: 2,
    printBar: false,
    title: gettext('CPU频率'),
    textField: 'cpusensors',
    renderer:function(value){
        var f0 = value.match(/CPU MHz.*?([\d]+)/)[1];
        var f1 = value.match(/CPU min MHz.*?([\d]+)/)[1];
        var f2 = value.match(/CPU max MHz.*?([\d]+)/)[1];
        return `实时: ${f0} MHz || 最小: ${f1} MHz | 最大: ${f2} MHz `
    }
},

然后重启pve服务即可。systemctl restart pveproxy

pveStatus