[OmniOS-discuss] COMMUNITY EXERCISE: illumos 7590
Dan McDonald
danmcd at omniti.com
Tue May 16 21:22:45 UTC 2017
THIS IS A LONG EMAIL WITH SOME COMMAND TRANSCRIPTS! PLEASE READ IT ALL IF YOU CARE ABOUT PUSHING OmniOS FORWARD! LOOK FOR "YOUR HOMEWORK" to see the actual exercises for the community.
. . .
Earlier today, I RTI-approved and pushed illumos bug 7590:
https://www.illumos.org/issues/7590
It was reported by the Samba team, and it appears to be a barrier to using Samba on illumos distros. Were I moving forward with OmniOS maintenance, I'd tag this one as a backport candidate for r151022.
INSTEAD, I'm going to start the ball rolling as publicly as possible about how one would pick a post-release illumos bugfix for backporting into a stable or even LTS release.
So let's outline the steps for backporting an illumos fix into an existing release. I picked this one because it's HARD. It spreads its tentacles across multiple packages, and even changes a public header file!
0.) While doing the steps below, keep in mind whether or not this is really worth tagging by itself, or whether or not it should wait until other updates also seem to be backport candidates.
YOUR HOMEWORK PART 0: Figure this out.
1.) Get the master branch of illumos-omnios merged with the very latest illumos-gate, which will bring the bug into illumos-omnios:master.
That is the easiest part of this. Going forward, illumos-omnios:master should be periodically updated with "git merge upstream" commands after illumos-omnios:upstream has been resynched with illumos-gate. This is all detailed in the first part of this page:
https://omnios.omniti.com/wiki.php/Maintainers
Since this is easy, I'm going ahead and doing it:
========================
bloody(~)[0]% illumos-update
HEY -- pay attention in case any of these FUBAR somehow.
~/ws/illumos-gate ~
Fetching origin
remote: Counting objects: 163, done.
remote: Compressing objects: 100% (66/66), done.
remote: Total 163 (delta 102), reused 77 (delta 77), pack-reused 13
Receiving objects: 100% (163/163), 173.54 KiB | 0 bytes/s, done.
Resolving deltas: 100% (102/102), completed with 76 local objects.
From https://github.com/illumos/illumos-gate
49bf4c2bcf..f012ee0c3d master -> origin/master
Updating 49bf4c2bcf..f012ee0c3d
Fast-forward
exception_lists/cstyle | 4 +
usr/src/lib/libdscfg/common/cfg.c | 290 +++++++++---------
usr/src/lib/libndmp/common/libndmp_prop.c | 3 +-
.../lib/print/libpapi-common/common/attribute.c | 171 +++++------
usr/src/man/man3nsl/xdr_admin.3nsl | 21 +-
usr/src/pkg/manifests/system-test-ostest.mf | 9 +-
usr/src/test/os-tests/runfiles/default.run | 8 +
usr/src/test/os-tests/tests/Makefile | 4 +-
usr/src/test/os-tests/tests/sockfs/Makefile | 54 ++++
usr/src/test/os-tests/tests/sockfs/conn.c | 220 ++++++++++++++
usr/src/test/os-tests/tests/sockfs/dgram.c | 182 ++++++++++++
usr/src/test/os-tests/tests/sockfs/drop_priv.c | 330 +++++++++++++++++++++
usr/src/test/os-tests/tests/sockfs/sockpair.c | 168 +++++++++++
usr/src/test/os-tests/tests/stress/Makefile | 42 +++
usr/src/test/os-tests/tests/stress/dladm-kstat.sh | 67 +++++
usr/src/uts/common/avs/ns/sdbc/sd_bio.c | 11 +-
usr/src/uts/common/fs/autofs/auto_vfsops.c | 5 +-
usr/src/uts/common/fs/autofs/auto_vnops.c | 4 +-
usr/src/uts/common/fs/dcfs/dc_vnops.c | 6 +-
usr/src/uts/common/fs/dev/sdev_subr.c | 9 +-
usr/src/uts/common/fs/dev/sdev_vfsops.c | 9 +-
usr/src/uts/common/fs/devfs/devfs_vnops.c | 7 +-
usr/src/uts/common/fs/dnlc.c | 17 +-
usr/src/uts/common/fs/doorfs/door_vnops.c | 23 +-
usr/src/uts/common/fs/fd/fdops.c | 22 +-
usr/src/uts/common/fs/fifofs/fifovnops.c | 40 ++-
usr/src/uts/common/fs/gfs.c | 7 +-
usr/src/uts/common/fs/hsfs/hsfs_vnops.c | 209 ++++---------
usr/src/uts/common/fs/lofs/lofs_subr.c | 11 +-
usr/src/uts/common/fs/namefs/namevfs.c | 4 +-
usr/src/uts/common/fs/namefs/namevno.c | 36 +--
usr/src/uts/common/fs/nfs/nfs4_client.c | 12 +-
usr/src/uts/common/fs/nfs/nfs4_rnode.c | 13 +-
usr/src/uts/common/fs/nfs/nfs4_shadow.c | 5 +-
usr/src/uts/common/fs/nfs/nfs_subr.c | 14 +-
usr/src/uts/common/fs/pcfs/pc_node.c | 5 +-
usr/src/uts/common/fs/pcfs/pc_vnops.c | 3 +-
usr/src/uts/common/fs/proc/prvnops.c | 3 +-
usr/src/uts/common/fs/smbclnt/smbfs/smbfs_subr2.c | 15 +-
usr/src/uts/common/fs/sockfs/sockcommon_vnops.c | 9 +-
usr/src/uts/common/fs/sockfs/socksubr.c | 16 +-
usr/src/uts/common/fs/sockfs/socktpi.c | 234 +++++++++++----
usr/src/uts/common/fs/sockfs/socktpi.h | 9 +-
usr/src/uts/common/fs/specfs/specvnops.c | 7 +-
usr/src/uts/common/fs/tmpfs/tmp_vnops.c | 3 +-
usr/src/uts/common/fs/udfs/udf_inode.c | 21 +-
usr/src/uts/common/fs/ufs/ufs_inode.c | 10 +-
usr/src/uts/common/fs/ufs/ufs_thread.c | 13 +-
usr/src/uts/common/fs/vnode.c | 13 +-
usr/src/uts/common/fs/xattr.c | 25 +-
usr/src/uts/common/fs/zfs/zfs_ctldir.c | 4 +-
usr/src/uts/common/fs/zfs/zfs_vnops.c | 6 +-
usr/src/uts/common/fs/zfs/zfs_znode.c | 4 +-
usr/src/uts/common/io/dls/dls_mgmt.c | 60 ++--
usr/src/uts/common/os/contract.c | 5 +-
usr/src/uts/common/os/driver.c | 11 +-
usr/src/uts/common/sys/fs/sdev_impl.h | 9 +-
usr/src/uts/common/sys/socket.h | 5 +-
usr/src/uts/common/sys/vnode.h | 39 ++-
59 files changed, 1855 insertions(+), 711 deletions(-)
create mode 100644 usr/src/test/os-tests/tests/sockfs/Makefile
create mode 100644 usr/src/test/os-tests/tests/sockfs/conn.c
create mode 100644 usr/src/test/os-tests/tests/sockfs/dgram.c
create mode 100644 usr/src/test/os-tests/tests/sockfs/drop_priv.c
create mode 100644 usr/src/test/os-tests/tests/sockfs/sockpair.c
create mode 100644 usr/src/test/os-tests/tests/stress/Makefile
create mode 100755 usr/src/test/os-tests/tests/stress/dladm-kstat.sh
~
~/ws/illumos-omnios ~
Fetching origin
From https://github.com/omniti-labs/illumos-omnios
3f784048a3..8b31933c62 master -> origin/master
5806135a39..49bf4c2bcf upstream -> origin/upstream
7f03e7512c..f6eee9d5d9 upstream_joyent -> origin/upstream_joyent
Already up-to-date.
Switched to branch 'upstream'
Your branch is up-to-date with 'origin/upstream'.
remote: Counting objects: 163, done.
remote: Compressing objects: 100% (51/51), done.
remote: Total 163 (delta 132), reused 133 (delta 102)
Receiving objects: 100% (163/163), 27.33 KiB | 0 bytes/s, done.
Resolving deltas: 100% (132/132), completed with 94 local objects.
From /export/home/danmcd/ws/illumos-gate
* branch HEAD -> FETCH_HEAD
Updating 49bf4c2bcf..f012ee0c3d
Fast-forward
exception_lists/cstyle | 4 +
usr/src/lib/libdscfg/common/cfg.c | 290 +++++++++---------
usr/src/lib/libndmp/common/libndmp_prop.c | 3 +-
.../lib/print/libpapi-common/common/attribute.c | 171 +++++------
usr/src/man/man3nsl/xdr_admin.3nsl | 21 +-
usr/src/pkg/manifests/system-test-ostest.mf | 9 +-
usr/src/test/os-tests/runfiles/default.run | 8 +
usr/src/test/os-tests/tests/Makefile | 4 +-
usr/src/test/os-tests/tests/sockfs/Makefile | 54 ++++
usr/src/test/os-tests/tests/sockfs/conn.c | 220 ++++++++++++++
usr/src/test/os-tests/tests/sockfs/dgram.c | 182 ++++++++++++
usr/src/test/os-tests/tests/sockfs/drop_priv.c | 330 +++++++++++++++++++++
usr/src/test/os-tests/tests/sockfs/sockpair.c | 168 +++++++++++
usr/src/test/os-tests/tests/stress/Makefile | 42 +++
usr/src/test/os-tests/tests/stress/dladm-kstat.sh | 67 +++++
usr/src/uts/common/avs/ns/sdbc/sd_bio.c | 11 +-
usr/src/uts/common/fs/autofs/auto_vfsops.c | 5 +-
usr/src/uts/common/fs/autofs/auto_vnops.c | 4 +-
usr/src/uts/common/fs/dcfs/dc_vnops.c | 6 +-
usr/src/uts/common/fs/dev/sdev_subr.c | 9 +-
usr/src/uts/common/fs/dev/sdev_vfsops.c | 9 +-
usr/src/uts/common/fs/devfs/devfs_vnops.c | 7 +-
usr/src/uts/common/fs/dnlc.c | 17 +-
usr/src/uts/common/fs/doorfs/door_vnops.c | 23 +-
usr/src/uts/common/fs/fd/fdops.c | 22 +-
usr/src/uts/common/fs/fifofs/fifovnops.c | 40 ++-
usr/src/uts/common/fs/gfs.c | 7 +-
usr/src/uts/common/fs/hsfs/hsfs_vnops.c | 209 ++++---------
usr/src/uts/common/fs/lofs/lofs_subr.c | 11 +-
usr/src/uts/common/fs/namefs/namevfs.c | 4 +-
usr/src/uts/common/fs/namefs/namevno.c | 36 +--
usr/src/uts/common/fs/nfs/nfs4_client.c | 12 +-
usr/src/uts/common/fs/nfs/nfs4_rnode.c | 13 +-
usr/src/uts/common/fs/nfs/nfs4_shadow.c | 5 +-
usr/src/uts/common/fs/nfs/nfs_subr.c | 14 +-
usr/src/uts/common/fs/pcfs/pc_node.c | 5 +-
usr/src/uts/common/fs/pcfs/pc_vnops.c | 3 +-
usr/src/uts/common/fs/proc/prvnops.c | 3 +-
usr/src/uts/common/fs/smbclnt/smbfs/smbfs_subr2.c | 15 +-
usr/src/uts/common/fs/sockfs/sockcommon_vnops.c | 9 +-
usr/src/uts/common/fs/sockfs/socksubr.c | 16 +-
usr/src/uts/common/fs/sockfs/socktpi.c | 234 +++++++++++----
usr/src/uts/common/fs/sockfs/socktpi.h | 9 +-
usr/src/uts/common/fs/specfs/specvnops.c | 7 +-
usr/src/uts/common/fs/tmpfs/tmp_vnops.c | 3 +-
usr/src/uts/common/fs/udfs/udf_inode.c | 21 +-
usr/src/uts/common/fs/ufs/ufs_inode.c | 10 +-
usr/src/uts/common/fs/ufs/ufs_thread.c | 13 +-
usr/src/uts/common/fs/vnode.c | 13 +-
usr/src/uts/common/fs/xattr.c | 25 +-
usr/src/uts/common/fs/zfs/zfs_ctldir.c | 4 +-
usr/src/uts/common/fs/zfs/zfs_vnops.c | 6 +-
usr/src/uts/common/fs/zfs/zfs_znode.c | 4 +-
usr/src/uts/common/io/dls/dls_mgmt.c | 60 ++--
usr/src/uts/common/os/contract.c | 5 +-
usr/src/uts/common/os/driver.c | 11 +-
usr/src/uts/common/sys/fs/sdev_impl.h | 9 +-
usr/src/uts/common/sys/socket.h | 5 +-
usr/src/uts/common/sys/vnode.h | 39 ++-
59 files changed, 1855 insertions(+), 711 deletions(-)
create mode 100644 usr/src/test/os-tests/tests/sockfs/Makefile
create mode 100644 usr/src/test/os-tests/tests/sockfs/conn.c
create mode 100644 usr/src/test/os-tests/tests/sockfs/dgram.c
create mode 100644 usr/src/test/os-tests/tests/sockfs/drop_priv.c
create mode 100644 usr/src/test/os-tests/tests/sockfs/sockpair.c
create mode 100644 usr/src/test/os-tests/tests/stress/Makefile
create mode 100755 usr/src/test/os-tests/tests/stress/dladm-kstat.sh
Checking out files: 100% (811/811), done.
Switched to branch 'master'
Your branch is up-to-date with 'origin/master'.
So far so good... I'm going to give you a chance to ^C out of here before
I 'git merge upstream'. At this point, illumos-omnios's 'upstream'
branch is up to date with illumos-gate, but the master branch is checked
out.
Press RETURN to continue.
Auto-merging usr/src/uts/common/sys/vnode.h
CONFLICT (content): Merge conflict in usr/src/uts/common/sys/vnode.h
Auto-merging usr/src/uts/common/sys/socket.h
Auto-merging usr/src/uts/common/os/contract.c
Auto-merging usr/src/uts/common/io/dls/dls_mgmt.c
Auto-merging usr/src/uts/common/fs/zfs/zfs_vnops.c
Auto-merging usr/src/uts/common/fs/vnode.c
Auto-merging usr/src/uts/common/fs/tmpfs/tmp_vnops.c
Auto-merging usr/src/uts/common/fs/sockfs/socksubr.c
CONFLICT (content): Merge conflict in usr/src/uts/common/fs/sockfs/socksubr.c
Auto-merging usr/src/uts/common/fs/proc/prvnops.c
Auto-merging usr/src/uts/common/fs/pcfs/pc_vnops.c
Auto-merging usr/src/uts/common/fs/dev/sdev_subr.c
Auto-merging usr/src/test/os-tests/tests/Makefile
CONFLICT (content): Merge conflict in usr/src/test/os-tests/tests/Makefile
Auto-merging usr/src/test/os-tests/runfiles/default.run
CONFLICT (content): Merge conflict in usr/src/test/os-tests/runfiles/default.run
Auto-merging usr/src/pkg/manifests/system-test-ostest.mf
CONFLICT (content): Merge conflict in usr/src/pkg/manifests/system-test-ostest.mf
Automatic merge failed; fix conflicts and then commit the result.
~
bloody(~)[0]%
========================
OH SNAP! Looks like it's not so easy. I see five conflicts, and all of them I'll bet are related to LX-brand changes we imported into OmniOS. Now I'm going to resolve the conflicts, and summarize them here.
* Two sets of new tests from -gate collided with a timer test that was brought in with LX.
* Simple copyright-collisions for the two in-kernel. (Both contain LX enhancements, so they have more recent Joyent copyrights that what's in -gate).
The tests are collidy-enough where I'm running an illumos-omnios build before I push them. For now, let's assume it works and I push these upstream to the illumos-omnios repo.
2.) Cherrypick the commit from master into the r151022 branch.
Like with merging from upstream, there's a good chance we'll have collisions, and they may very well be the same as the merge from -gate. The "--strategy=3-way" flag may be useful here, as you'll get collision indicators. Since I'm not done yet with the illumos-omnios test build, I'll hold off.
3.) Build r151022 to make sure it actually builds!
With sufficient hardware, this is less than an hour using ALL the build flags you need.
4.) Figure out either which packages need updating, OR resign yourself to a full-illumos update.
The full-illumos update is easier, but requires FAR more package transfers & signatures. You basically run the omnios-build/build/illumos/build.sh script with a PREBUILT_ILLUMOS set to whatever you did in step 3, setting PKGSRVR to somewhere nice (make sure omnios-build is on branch r151022):
pkgrepo create -s /tmp/staging.repo
pkgrepo add-publisher -s /tmp/staging.repo omnios
cd $PATH/omnios-build/build/illumos
PREBUILT_ILLUMOS=$PATH/build/illumos-omnios PKGSRVR=/tmp/staging.repo ./build.sh -lb
Figuring out which packages requires you look at the commit to see what packages get touched. Also, you need to figure out if this is a require-reboot update or not, and finally, you need to ask if it's worth respinning the "uname -v" string, which requires system/kernel/platform to be updated as well.
YOUR HOMEWORK PART 1: Figure out what packages illumos 7590 touches.
5.) Get packages signed.
6.) Push packages out.
For now, these involves OmniTI's help. I'm sure that won't be a problem.
7.) Decide if you need to build media again.
IMHO for this one you don't. But...
YOUR HOMEWORK PART 2: Figure out if illumos 7590 is worth a media spin, or just tell users to "pkg update"?
8.) Build media.
You can only do this after the packages are pushed on to the repo servers. Luckily, it's pretty easy these days, esp. if you have your PREBUILT_ILLUMOS from step 3:
cd $PATH/kayak
(Make sure you're on branch r151022)
sudo zfs destroy -R rpool/kayak_image (NOTE: Can use different one)
sudo zfs create rpool/kayak_image
sudo gmake BUILDSEND=rpool/kayak_image PREBUILT_ILLUMOS=$PATH/build/illumos-omnios install-iso
The media will all be in /rpool/kayak_image.
Ask away if you've questions or (more likely) if I missed something.
Thanks,
Dan
More information about the OmniOS-discuss
mailing list