Comments

• 

  about 1 year ago by zzhou | Reply

  ``` * Exercise-1: Lunch multiple NFS docker instances directly by systemd inside containers

  1. tumbleweed status target : 20231101

  Dockerfile:

  FROM opensuse/tumbleweed RUN zypper -n install systemd nfs-kernel-server vim iproute2 iputils pacemaker-remote gawk which RUN systemctl enable nfs-server RUN echo "/srv/nfs/share *(rw)" > /etc/exports CMD ["/usr/lib/systemd/systemd", "--system"]

  1. docker build -t nfsserver .

  runnfsserverdocker () { i=$1 # eg. i=1; N=nfsserver; h=$N-$i; \ docker run -v /srv/nfs${i}/state:/var/lib/nfs \ -v /srv/nfs${i}/share:/srv/nfs/share \ -it --privileged --name=$h -h=$h $N & } runnfsserverdocker 1 runnfsserverdocker 2

  1. verify two nfsserver docker instances ipnfsserver1=docker inspect -f '{{range.NetworkSettings.Networks}}{{.IPAddress}}{{end}}' nfsserver-1; echo $ip1 ipnfsserver2=docker inspect -f '{{range.NetworkSettings.Networks}}{{.IPAddress}}{{end}}' nfsserver-2; echo $ip2 showmount -e $ipnfsserver1 showmount -e $ipnfsserver2

  tws-1:~ # showmount -e $ipnfsserver1 Export list for 172.17.0.3: /srv/nfs/share * tws-1:~ # showmount -e $ipnfsserver2 Export list for 172.17.0.4: /srv/nfs/share *

  • Exercise-2: Lunch NFS server docker directly by pacemaker-remote inside containers
    

  1. Add pacemaker docker bundle into CIB.xml TODO: FEAT: crmsh does not support container yet

  • Summary:
    

  Unfortunately in the end, this reveals a significant issue as the "show stopper" because pcmk-init for pacemaker-remote inside the container conflicts with systemd, both of them require PID=1.

  Open questions: - what's the purpose of pcmk-init in the bundle container? - If possible let the pacemaker bundle container still run systemd? - Is there any solid/stable approach to run nfsserver containers without systemd?

  ```

×
Edit Comment 7718
# A First Level Header
## A Second Level Header

Use one asterisk to *emphasize*

Use two asterisks for **strong emphasis**

- Use hyphens
- for unordereed
- lists

This is an [link to example.com](http://example.com/)

This is an image ![an openSUSE geeko icon](https://en.opensuse.org/images/d/d0/Icon-distribution.png)

This is a user link @hans

This is a project link hw#some-cool-title
More Complex Markdown Help
Formatting Help
• Edit
• Preview
``` * Exercise-1: Lunch multiple NFS docker instances directly by systemd inside containers 1. tumbleweed status target : 20231101 Dockerfile: FROM opensuse/tumbleweed RUN zypper -n install systemd nfs-kernel-server vim iproute2 iputils pacemaker-remote gawk which RUN systemctl enable nfs-server RUN echo "/srv/nfs/share *(rw)" > /etc/exports CMD ["/usr/lib/systemd/systemd", "--system"] 2. docker build -t nfsserver . run_nfsserver_docker () { i=$1 # eg. i=1; N=nfsserver; h=$N-$i; \ docker run -v /srv/nfs${i}/state:/var/lib/nfs \ -v /srv/nfs${i}/share:/srv/nfs/share \ -it --privileged --name=$h -h=$h $N & } run_nfsserver_docker 1 run_nfsserver_docker 2 3. verify two nfsserver docker instances ip_nfsserver_1=`docker inspect -f '{{range.NetworkSettings.Networks}}{{.IPAddress}}{{end}}' nfsserver-1`; echo $ip_1 ip_nfsserver_2=`docker inspect -f '{{range.NetworkSettings.Networks}}{{.IPAddress}}{{end}}' nfsserver-2`; echo $ip_2 showmount -e $ip_nfsserver_1 showmount -e $ip_nfsserver_2 tws-1:~ # showmount -e $ip_nfsserver_1 Export list for 172.17.0.3: /srv/nfs/share * tws-1:~ # showmount -e $ip_nfsserver_2 Export list for 172.17.0.4: /srv/nfs/share * * Exercise-2: Lunch NFS server docker directly by pacemaker-remote inside containers 4. Add pacemaker docker bundle into CIB.xml TODO: FEAT: crmsh does not support container yet <bundle id="nfs"> <docker image="nfsserver"/> <network ip-range-start="172.17.0.100" host-netmask="16"/> <storage> <storage-mapping id="nfs-state" source-dir-root="/srv/nfs-state" target-dir="/var/lib/nfs" options="rw"/> <storage-mapping id="nfs-share" source-dir-root="/srv/nfs-share" target-dir="/srv/nfs/share" options="rw"/> </storage> <primitive id="nfsserver001" class="ocf" provider="heartbeat" type="nfsserver"/> </bundle> * Summary: Unfortunately in the end, this reveals a significant issue as the "show stopper" because pcmk-init for pacemaker-remote inside the container conflicts with systemd, both of them require PID=1. Open questions: - what's the purpose of pcmk-init in the bundle container? - If possible let the pacemaker bundle container still run systemd? - Is there any solid/stable approach to run nfsserver containers without systemd? ```
 
×
Reply to zzhou
``` * Exercise-1: Lunch multiple NFS docker instances directly by systemd inside containers
1. tumbleweed status target : 20231101
Dockerfile:
FROM opensuse/tumbleweed RUN zypper -n install systemd nfs-kernel-server vim iproute2 iputils pacemaker-remote gawk which RUN systemctl enable nfs-server RUN echo "/srv/nfs/share *(rw)" > /etc/exports CMD ["/usr/lib/systemd/systemd", "--system"]
1. docker build -t nfsserver .
runnfsserverdocker () { i=$1 # eg. i=1; N=nfsserver; h=$N-$i; \ docker run -v /srv/nfs${i}/state:/var/lib/nfs \ -v /srv/nfs${i}/share:/srv/nfs/share \ -it --privileged --name=$h -h=$h $N & } runnfsserverdocker 1 runnfsserverdocker 2
1. verify two nfsserver docker instances ipnfsserver1=docker inspect -f '{{range.NetworkSettings.Networks}}{{.IPAddress}}{{end}}' nfsserver-1; echo $ip1 ipnfsserver2=docker inspect -f '{{range.NetworkSettings.Networks}}{{.IPAddress}}{{end}}' nfsserver-2; echo $ip2 showmount -e $ipnfsserver1 showmount -e $ipnfsserver2
tws-1:~ # showmount -e $ipnfsserver1 Export list for 172.17.0.3: /srv/nfs/share * tws-1:~ # showmount -e $ipnfsserver2 Export list for 172.17.0.4: /srv/nfs/share *
• Exercise-2: Lunch NFS server docker directly by pacemaker-remote inside containers
  
1. Add pacemaker docker bundle into CIB.xml TODO: FEAT: crmsh does not support container yet
• Summary:
  
Unfortunately in the end, this reveals a significant issue as the "show stopper" because pcmk-init for pacemaker-remote inside the container conflicts with systemd, both of them require PID=1.
Open questions: - what's the purpose of pcmk-init in the bundle container? - If possible let the pacemaker bundle container still run systemd? - Is there any solid/stable approach to run nfsserver containers without systemd?
```
---
# A First Level Header
## A Second Level Header

Use one asterisk to *emphasize*

Use two asterisks for **strong emphasis**

- Use hyphens
- for unordereed
- lists

This is an [link to example.com](http://example.com/)

This is an image ![an openSUSE geeko icon](https://en.opensuse.org/images/d/d0/Icon-distribution.png)

This is a user link @hans

This is a project link hw#some-cool-title
More Complex Markdown Help
Formatting Help
• Edit
• Preview
 
• 

  11 months ago by zzhou | Reply

  ``` Back on this topic and update some major progress since hackweek 23.

  In summary, the show-stopper issue in the past has been addressed. As a result, nfsserver can now operate within the Pacemaker bundle containers and can be distributed across the cluster nodes. The successful mounting of nfsserver exports on various cluster nodes using NFSv3 and v4 protocols has been confirmed.

  The ongoing challenge lies with the showmount operation, specifically in the context of the NFS protocol GETADDR operation, which is currently not functioning correctly. Unfortunately, a resolution for this issue has not been identified yet.

  The sample configuration is provided below. Some refinement may still be necessary, and adjustments might be required to enhance certain subtle NFS functionalities. [1] for Dockerfile, [2] for crm configure show

  [1] Dockerfile

  FROM opensuse/tumbleweed RUN zypper -n install systemd nfs-kernel-server vim iproute2 iputils pacemaker-remote gawk which

  RUN echo -e "[Unit]\nRequires=pacemakerremote.service\nAfter=pacemakerremote.service\nRequires=nfs-server.service\nAfter=nfs-server.service" > /usr/lib/systemd/system/runpcmkremoteandnfs_server.target RUN echo -e "[Service]\nExecStartPre=/usr/bin/mkdir -p /var/lib/nfs/sm /var/lib/nfs/sm.bak" > /usr/lib/systemd/system/nfs-server.service.d/10-prepare-dirs.conf

  RUN sed -e 's/STATDPORT=.*/STATDPORT="662"/' -i /etc/sysconfig/nfs RUN sed -e 's/LOCKDTCPPORT=.*/LOCKDTCPPORT="32768"/' -i /etc/sysconfig/nfs RUN sed -e 's/LOCKDUDPPORT=.*/LOCKDUDPPORT="32768"/' -i /etc/sysconfig/nfs

  CMD ["/usr/lib/systemd/systemd", "--system"]

  [2] crm configure show

  primitive drbd1 ocf:linbit:drbd \ params drbdresource=nfsserver1 \ op monitor interval=15 role=Promoted timeout=20 \ op monitor interval=30 role=Unpromoted timeout=20 \ op start timeout=240 interval=0s \ op promote timeout=90 interval=0s \ op demote timeout=90 interval=0s \ op stop timeout=100 interval=0s primitive exportfs1 exportfs \ params directory="/srv/nfs/share" options="rw,mountpoint" clientspec="*" fsid=0 \ op monitor interval=30s timeout=40s \ op start timeout=60s interval=0s \ op stop timeout=120s interval=0s primitive fs1 Filesystem \ params device="/dev/drbd1" directory="/srv/nfs1" fstype=ext4 \ op monitor interval=30s timeout=40s \ op start timeout=60s interval=0s \ op stop timeout=60s interval=0s bundle nfsserver1 \ docker image=nfsserver options="--privileged --stop-signal SIGRTMIN+3" run-command="/usr/lib/systemd/systemd --system --unit=runpcmkremoteandnfsserver.target" \ network ip-range-start=192.168.1.200 port-mapping id=nfs1portsunrpc port=111 port-mapping id=nfs1portdata port=2049 port-mapping id=nfs1portrpcmount port=20048 port-mapping id=nfs1portstatd port=662 port-mapping id=nfs1portlockd-tcpudp port=32768 \ storage storage-mapping id=nfs1-state source-dir="/srv/nfs1/state" target-dir="/var/lib/nfs" options=rw storage-mapping id=nfs1-share source-dir="/srv/nfs1/share" target-dir="/srv/nfs/share" options=rw \ meta target-role=Started \ primitive exportfs1 clone drbd-nfs1 drbd1 \ meta promotable=true promoted-max=1 promoted-node-max=1 clone-max=2 clone-node-max=1 notify=true interleave=true colocation col-nfs1-on-drbd inf: nfsserver1 fs1 drbd-nfs1:Promoted order o-drbd-before-nfs1 Mandatory: drbd-nfs1:promote fs1:start nfsserver1

  ```

×
Edit Comment 7843
# A First Level Header
## A Second Level Header

Use one asterisk to *emphasize*

Use two asterisks for **strong emphasis**

- Use hyphens
- for unordereed
- lists

This is an [link to example.com](http://example.com/)

This is an image ![an openSUSE geeko icon](https://en.opensuse.org/images/d/d0/Icon-distribution.png)

This is a user link @hans

This is a project link hw#some-cool-title
More Complex Markdown Help
Formatting Help
• Edit
• Preview
``` Back on this topic and update some major progress since hackweek 23. In summary, the show-stopper issue in the past has been addressed. As a result, nfsserver can now operate within the Pacemaker bundle containers and can be distributed across the cluster nodes. The successful mounting of nfsserver exports on various cluster nodes using NFSv3 and v4 protocols has been confirmed. The ongoing challenge lies with the `showmount` operation, specifically in the context of the NFS protocol GETADDR operation, which is currently not functioning correctly. Unfortunately, a resolution for this issue has not been identified yet. The sample configuration is provided below. Some refinement may still be necessary, and adjustments might be required to enhance certain subtle NFS functionalities. [1] for Dockerfile, [2] for `crm configure show` [1] Dockerfile FROM opensuse/tumbleweed RUN zypper -n install systemd nfs-kernel-server vim iproute2 iputils pacemaker-remote gawk which RUN echo -e "[Unit]\nRequires=pacemaker_remote.service\nAfter=pacemaker_remote.service\nRequires=nfs-server.service\nAfter=nfs-server.service" > /usr/lib/systemd/system/run_pcmk_remote_and_nfs_server.target RUN echo -e "[Service]\nExecStartPre=/usr/bin/mkdir -p /var/lib/nfs/sm /var/lib/nfs/sm.bak" > /usr/lib/systemd/system/nfs-server.service.d/10-prepare-dirs.conf RUN sed -e 's/STATD_PORT=.*/STATD_PORT="662"/' -i /etc/sysconfig/nfs RUN sed -e 's/LOCKD_TCPPORT=.*/LOCKD_TCPPORT="32768"/' -i /etc/sysconfig/nfs RUN sed -e 's/LOCKD_UDPPORT=.*/LOCKD_UDPPORT="32768"/' -i /etc/sysconfig/nfs CMD ["/usr/lib/systemd/systemd", "--system"] [2] crm configure show primitive drbd1 ocf:linbit:drbd \ params drbd_resource=nfsserver1 \ op monitor interval=15 role=Promoted timeout=20 \ op monitor interval=30 role=Unpromoted timeout=20 \ op start timeout=240 interval=0s \ op promote timeout=90 interval=0s \ op demote timeout=90 interval=0s \ op stop timeout=100 interval=0s primitive exportfs1 exportfs \ params directory="/srv/nfs/share" options="rw,mountpoint" clientspec="*" fsid=0 \ op monitor interval=30s timeout=40s \ op start timeout=60s interval=0s \ op stop timeout=120s interval=0s primitive fs1 Filesystem \ params device="/dev/drbd1" directory="/srv/nfs1" fstype=ext4 \ op monitor interval=30s timeout=40s \ op start timeout=60s interval=0s \ op stop timeout=60s interval=0s bundle nfsserver1 \ docker image=nfsserver options="--privileged --stop-signal SIGRTMIN+3" run-command="/usr/lib/systemd/systemd --system --unit=run_pcmk_remote_and_nfs_server.target" \ network ip-range-start=192.168.1.200 port-mapping id=nfs1_port_sunrpc port=111 port-mapping id=nfs1_port_data port=2049 port-mapping id=nfs1_port_rpcmount port=20048 port-mapping id=nfs1_port_statd port=662 port-mapping id=nfs1_port_lockd-tcpudp port=32768 \ storage storage-mapping id=nfs1-state source-dir="/srv/nfs1/state" target-dir="/var/lib/nfs" options=rw storage-mapping id=nfs1-share source-dir="/srv/nfs1/share" target-dir="/srv/nfs/share" options=rw \ meta target-role=Started \ primitive exportfs1 clone drbd-nfs1 drbd1 \ meta promotable=true promoted-max=1 promoted-node-max=1 clone-max=2 clone-node-max=1 notify=true interleave=true colocation col-nfs1-on-drbd inf: nfsserver1 fs1 drbd-nfs1:Promoted order o-drbd-before-nfs1 Mandatory: drbd-nfs1:promote fs1:start nfsserver1 ```
 
×
Reply to zzhou
``` Back on this topic and update some major progress since hackweek 23.
In summary, the show-stopper issue in the past has been addressed. As a result, nfsserver can now operate within the Pacemaker bundle containers and can be distributed across the cluster nodes. The successful mounting of nfsserver exports on various cluster nodes using NFSv3 and v4 protocols has been confirmed.
The ongoing challenge lies with the showmount operation, specifically in the context of the NFS protocol GETADDR operation, which is currently not functioning correctly. Unfortunately, a resolution for this issue has not been identified yet.
The sample configuration is provided below. Some refinement may still be necessary, and adjustments might be required to enhance certain subtle NFS functionalities. [1] for Dockerfile, [2] for crm configure show
[1] Dockerfile
FROM opensuse/tumbleweed RUN zypper -n install systemd nfs-kernel-server vim iproute2 iputils pacemaker-remote gawk which
RUN echo -e "[Unit]\nRequires=pacemakerremote.service\nAfter=pacemakerremote.service\nRequires=nfs-server.service\nAfter=nfs-server.service" > /usr/lib/systemd/system/runpcmkremoteandnfs_server.target RUN echo -e "[Service]\nExecStartPre=/usr/bin/mkdir -p /var/lib/nfs/sm /var/lib/nfs/sm.bak" > /usr/lib/systemd/system/nfs-server.service.d/10-prepare-dirs.conf
RUN sed -e 's/STATDPORT=.*/STATDPORT="662"/' -i /etc/sysconfig/nfs RUN sed -e 's/LOCKDTCPPORT=.*/LOCKDTCPPORT="32768"/' -i /etc/sysconfig/nfs RUN sed -e 's/LOCKDUDPPORT=.*/LOCKDUDPPORT="32768"/' -i /etc/sysconfig/nfs
CMD ["/usr/lib/systemd/systemd", "--system"]
[2] crm configure show
primitive drbd1 ocf:linbit:drbd \ params drbdresource=nfsserver1 \ op monitor interval=15 role=Promoted timeout=20 \ op monitor interval=30 role=Unpromoted timeout=20 \ op start timeout=240 interval=0s \ op promote timeout=90 interval=0s \ op demote timeout=90 interval=0s \ op stop timeout=100 interval=0s primitive exportfs1 exportfs \ params directory="/srv/nfs/share" options="rw,mountpoint" clientspec="*" fsid=0 \ op monitor interval=30s timeout=40s \ op start timeout=60s interval=0s \ op stop timeout=120s interval=0s primitive fs1 Filesystem \ params device="/dev/drbd1" directory="/srv/nfs1" fstype=ext4 \ op monitor interval=30s timeout=40s \ op start timeout=60s interval=0s \ op stop timeout=60s interval=0s bundle nfsserver1 \ docker image=nfsserver options="--privileged --stop-signal SIGRTMIN+3" run-command="/usr/lib/systemd/systemd --system --unit=runpcmkremoteandnfsserver.target" \ network ip-range-start=192.168.1.200 port-mapping id=nfs1portsunrpc port=111 port-mapping id=nfs1portdata port=2049 port-mapping id=nfs1portrpcmount port=20048 port-mapping id=nfs1portstatd port=662 port-mapping id=nfs1portlockd-tcpudp port=32768 \ storage storage-mapping id=nfs1-state source-dir="/srv/nfs1/state" target-dir="/var/lib/nfs" options=rw storage-mapping id=nfs1-share source-dir="/srv/nfs1/share" target-dir="/srv/nfs/share" options=rw \ meta target-role=Started \ primitive exportfs1 clone drbd-nfs1 drbd1 \ meta promotable=true promoted-max=1 promoted-node-max=1 clone-max=2 clone-node-max=1 notify=true interleave=true colocation col-nfs1-on-drbd inf: nfsserver1 fs1 drbd-nfs1:Promoted order o-drbd-before-nfs1 Mandatory: drbd-nfs1:promote fs1:start nfsserver1
```
---
# A First Level Header
## A Second Level Header

Use one asterisk to *emphasize*

Use two asterisks for **strong emphasis**

- Use hyphens
- for unordereed
- lists

This is an [link to example.com](http://example.com/)

This is an image ![an openSUSE geeko icon](https://en.opensuse.org/images/d/d0/Icon-distribution.png)

This is a user link @hans

This is a project link hw#some-cool-title
More Complex Markdown Help
Formatting Help
• Edit
• Preview