Discussion:
[users] [OT] Analogies to "governance, " not so much "manufacturer" -- WAS: EPEL
Bryan J Smith
2011-06-26 12:07:10 UTC
Permalink
From: Yury V. Zaytsev <yury at shurup.com>
This question is in a very same vein as asking why would we have 5 major
car platform manufacturers producing mutually incompatible products,
which results in the duplication of the engineering efforts instead of
having one worldwide meta-manufacturer, that would produce the most
advanced car platforms in a truly efficient way.
Do you have an answer for that already?
Maybe it's because I'm a Libertarian-thinking American engineer, but I do not
believe that is an accurate analogy. Yes, I'm nitpicking ... ;)

Just want to point out the following reasons:
1. The assumption that multiple engineering teams are poor and less efficient
2. That a single entity or "bureau" would produce something more advanced
3. Many manufacturers actually use the exact same suppliers, with full
compatibility

If #1 is true, why not just have all engineers in a class work on a single
"super project" instead of challenging each other to see what ideas and designs
they could come up with? And #2, well, that's one of the greatest lessons in
engineering, as the concept of communism actually appeals to engineers, and most
of the Soviet Union was filled engineers in their political landscape. It's the
core belief that everything from marketing to microeconomics is a waste of time,
which is a clear west v. east engineering mentality, whereas the west learns
management and feasibility, and the east did not. And if you don't believe #3,
let's talk the "lowly" Corvette versus Ferrari sometime, and some of the rather
inexpensive, Anglo-American parts that do into high-end products of the latter.
Heck, even the Toyota mats came from an American manufacturer who spends 97% of
their time manufacturing for other makers (and they regularly had to point out
Toyota's requirements and specifications versus others, to show where the
differences came from).

A _better_ analogy would not be the independent, commercial car companies, but
more on the "goverance" side, conflicting North American transportation
standards with European Union, etc..., even inter-country or municipality
issues. Everything from headrests to lights, much more other aspects like
emissions and fuel -- including why Americans are seemingly "allergic" to diesel
(regulatory more than anything). Then you have both the manufacturers as well
as the consumers who have to deal with those conflicting standards, and redesign
even their own vehicles for different locales and regulatory requirements.
That's more of what is going on here, different standards, foci, maintainership,
leadership, etc... GM, Toyota, etc... have wildly differing divisions and even
names (e.g., GM's Holden in Australia), and if you ever have the chance to talk
to a chief engineer, you'll learn a lot about the issues. We can even talk
about the folly of assuming displacement means less fuel efficient, related
taxes and other things.

So does one just accept the Fedora Guidelines, Maintainer and Project governance
as "The" standard? That would certainly solve the problem, correct? How many
here would agree to that? ;)

Let's throw a more simplified auto analogy, racing. Here in the US, for sports
car racing (don't assume just because I'm an American that I follow NASCAR and
we Americans don't have anything but oval tracks ;) ), there is both IMSA
sponsored, ACO (EU) aligned, series like the American Le Mans Series (ALMS), and
more US-centric ISC run Grand Am program. Many would argue that the ACO is
already a real, international organization, with standards for prototype and
touring that everyone should follow. Others would complain there are some
serious differences in foci for racing between NA and EU, especially when it
comes to manfacturing-centric v. performance-centric. Both arguments have
merits. As much as the ACO has had quite a history, one can look back to the
'80s and Grand Am's predecessor in influencing major advances in everything from
ABS to traction control.


And that's before we consider the US' manufacturing-centric industry has
generally been allergic to formal sanction for fear of racing being seen as
dangerous and a liability for sales. Even GM was a major opponent (until
1998), and Ford only officially sanctioned it's '60s ACO teams because it was
spurned when it tried to by Ferrari (and even then it did not use its own
program, but looked to Shelby who looked outside of Ford). And even when
sanctioned today by the ACO, you have differences between NA and EU. ALMS is in
the bread'n butter consumer markets of the Americas, so it's clearly more
manufacturing-based. Even LMS (EU) is far more centric on the privateer teams
and drivers -- especially in FIA GT1 where few manufacturers go. I mean, if
you're a smaller design firm, who wants to challenge a deep pockets corporation
like a GM or Toyota when they have product branded manufacturer backed teams
that sell sub-$50K models that beat your 6-7 figure priced model continually?
Does your marketing absolutely no good.

Governance, not choice, is the issue. And playing fields are rarely level.
Different requirements tend to cause different needs. However, focusing on
co-existence and compatibility is a worthwhile endeavor, even if the governance
is not shared.
--
Bryan J Smith Professional, Technical Annoyance
Linked Profile: http://www.linkedin.com/in/bjsmith
Steve Huff
2011-06-26 12:20:28 UTC
Permalink
Post by Bryan J Smith
Bryan J Smith Professional, Technical Annoyance
yeah, i remember you from the CentOS list. if you would, please post lengthy and tangential content like this to a blog. thanks!

-steve
--
If this were played upon a stage now, I could condemn it as an improbable fiction. - Fabian, Twelfth Night, III,v
PGP 8477B706 (A92A 1F7E 6D76 16A0 BFF9 E61D AD54 0251 8477 B706)
Bryan J Smith
2011-06-26 15:00:29 UTC
Permalink
From: Steve Huff <shuff at vecna.org>
Post by Steve Huff
yeah, i remember you from the CentOS list. if you would, please post lengthy
and
tangential content like this to a blog. thanks!
I've been a lurker for years on this list, and I assist others off-list and
privately precisely because individual like yourself enjoy questioning me
on-list. Learned that years from the CentOS list. ;)

So one counter-analogy per year is hardly "troublesome," especially when I take
the time to preface it "[OT]" and other things, let alone didn't start the
original thread. But feel free to complain and blame me for the original thread
in its entirety.

I now return to lurking. ;)
--
Bryan J Smith Professional, Technical Annoyance
Linked Profile: http://www.linkedin.com/in/bjsmith
Bryan J Smith
2011-06-26 15:00:29 UTC
Permalink
From: Steve Huff <shuff at vecna.org>
Post by Steve Huff
yeah, i remember you from the CentOS list. if you would, please post lengthy
and
tangential content like this to a blog. thanks!
I've been a lurker for years on this list, and I assist others off-list and
privately precisely because individual like yourself enjoy questioning me
on-list. Learned that years from the CentOS list. ;)

So one counter-analogy per year is hardly "troublesome," especially when I take
the time to preface it "[OT]" and other things, let alone didn't start the
original thread. But feel free to complain and blame me for the original thread
in its entirety.

I now return to lurking. ;)
--
Bryan J Smith Professional, Technical Annoyance
Linked Profile: http://www.linkedin.com/in/bjsmith
Yury V. Zaytsev
2011-06-26 17:46:23 UTC
Permalink
Post by Bryan J Smith
Maybe it's because I'm a Libertarian-thinking American engineer, but I do not
believe that is an accurate analogy. Yes, I'm nitpicking ... ;)
I didn't mean to construct an accurate analogy. I wanted to point out
that "divergent development" exists just about everywhere outside of the
packaging world and there isn't always a way or even the need to fix
it.
Post by Bryan J Smith
However, focusing on co-existence and compatibility is a worthwhile
endeavor, even if the governance is not shared.
I think I was very explicit in my message, that packagers both behind
RepoForge and EPEL are interested in staying interoperable.
Post by Bryan J Smith
I've been a lurker for years on this list, and I assist others off-list and
privately precisely because individual like yourself enjoy questioning me
on-list. Learned that years from the CentOS list. ;)
It seems to me that somehow you've got it all wrong: assisting others
on-list is welcome and in fact is the purpose of this list, whereas
verbose off-topic discussions are not.

Therefore, I'd also appreciate the follow-up conversations on this topic
would taking place where they are more appropriate.
--
Sincerely yours,
Yury V. Zaytsev
Dag Wieers
2011-06-27 12:27:35 UTC
Permalink
Post by Bryan J Smith
From: Yury V. Zaytsev <yury at shurup.com>
This question is in a very same vein as asking why would we have 5 major
car platform manufacturers producing mutually incompatible products,
which results in the duplication of the engineering efforts instead of
having one worldwide meta-manufacturer, that would produce the most
advanced car platforms in a truly efficient way.
Do you have an answer for that already?
So does one just accept the Fedora Guidelines, Maintainer and Project governance
as "The" standard? That would certainly solve the problem, correct? How many
here would agree to that? ;)
Let me state again that when the Fedora project started off, RPMforge (or
rather my repository) already existed. I was very interested to join the
Fedora project on the premise that they would be doing RHEL packages as
well. Matthias (also part of the RPMforge project) did join the Fedora
project because he was mostly interested in Fedora packages anyway.

So there were a few problems that made us decide not to join the Fedora
project at that time:

- They were not interested in RHEL packages

- They were not interested in using macros to simplify supporting
multiple distributions

- They were not interested in using the %dist macro

You can find the heated discussions in a Fedora archive somewhere. The
reasoning was that they only had to provide packages for the latest Fedora
and any backward-compatibility would be more easily handled by forking
SPEC files.

While obviously we were looking at keeping packages updated for 3 to 4
distributions (RHEL2, RHEL3, RHEL4 and RHEL5) over a 7 year time-span. So
forking meant in the worst case making the same update to 4 SPEC files.

If you look at today:

- Fedora is doing RHEL packages

- Fedora is using macros to simplify maintaining packages

- Fedora has started using %dist macro in an incompatible way, so we
introduced the %dtag macro for our specific use-case

So I think it is a shame that the few voices from RPMforge got lost in the
very heated discussions of the crowd and at a certain point I simply quit
the list because you simply cannot argue a crowd that is unable to look at
things from a different perspective.

Sadly, the above led to the differences and incompatibilities we have
today. And I feel that there's nothing in my power that I could have done
differently to avoid it. And I have been very angry when Fedora started
doing EPEL and started introducing problems without tagging their packages
so they were easily identifiable as being EPEL's.

That said, for a large part I agree with the Fedora packaging guidelines
and Tom Callaway has been doing a great job at that. But the Fedora
packaging guidelines are *not* the reason why there are incompatibilities.
--
-- dag wieers, dag at wieers.com, http://dag.wieers.com/
-- dagit linux solutions, info at dagit.net, http://dagit.net/

[Any errors in spelling, tact or fact are transmission errors]
Bryan J Smith
2011-06-27 13:07:54 UTC
Permalink
You've always thought outside-the-box and too far ahead of a lot of people (and
I say this as the sincerest form of complement).

I've watched the Fedora Project's evolution over many years and while you might
have not been a part of the team or list directly, there's no denying your work
and approaches has indirectly and even directly impacted the project over the
years. I'm not afraid to say that in the least bit if anyone ever asks me, or
makes a statement about DAG, RPMforge, etc...

I mentioned Guidelines, Maintainer and Governance as a kind of "tri-fecta."
Back to my [counter-] analogy, it's more than just a choice (company) detail.
Different locales are trying to solve the same problems, and often use the same
guidelines (industry) and sources (companies) as other locales, but the
maintenance and governance differences end up making all the difference. In
other words, I've never seen it as a technical issue.

Just my observations, as a purposeful outsider to both RPMforge and Fedora/EPEL,
even if annoying to many. ;)




----- Original Message ----
From: Dag Wieers <dag at wieers.com>
Sent: Mon, June 27, 2011 8:27:35 AM

...
So there were a few problems that made us decide not to join the Fedora
project at that time:
- They were not interested in RHEL packages
- They were not interested in using macros to simplify supporting
multiple distributions
- They were not interested in using the %dist macro
...
If you look at today:
- Fedora is doing RHEL packages
- Fedora is using macros to simplify maintaining packages
- Fedora has started using %dist macro in an incompatible way, so we
introduced the %dtag macro for our specific use-case
...
That said, for a large part I agree with the Fedora packaging guidelines
and Tom Callaway has been doing a great job at that. But the Fedora
packaging guidelines are *not* the reason why there are incompatibilities.
Dag Wieers
2011-06-27 13:23:38 UTC
Permalink
Post by Bryan J Smith
You've always thought outside-the-box and too far ahead of a lot of people (and
I say this as the sincerest form of complement).
I've watched the Fedora Project's evolution over many years and while you might
have not been a part of the team or list directly, there's no denying your work
and approaches has indirectly and even directly impacted the project over the
years. I'm not afraid to say that in the least bit if anyone ever asks me, or
makes a statement about DAG, RPMforge, etc...
I mentioned Guidelines, Maintainer and Governance as a kind of "tri-fecta."
Back to my [counter-] analogy, it's more than just a choice (company) detail.
Different locales are trying to solve the same problems, and often use the same
guidelines (industry) and sources (companies) as other locales, but the
maintenance and governance differences end up making all the difference. In
other words, I've never seen it as a technical issue.
I think a big difference is the fact that we always preferred (mostly by
necessity) to have a few people doing lots of packages, rather then a lot
of people doing a few packages. This has the advantage that there's less
overhead to packaging and the process is light and flexible. But the
downside is that we cannot take on large projects, like compatibility with
eg. EPEL.

If you consider ATrpms, rpmfusion, and the myriad of other repositories,
compatibility with other repositories is a hard proposition, and even
tracking changes that may influence compatibility is something a small set
of people cannot undertake. I think there are other and better solutions
if we want the best experience for our users, but this project has more
important concerns than inter-repository compatibility IMO.

Compare it to world-peace or world-hunger, how much I like this to happen,
I am afraid I won't be able to make a big contribution to the overal
solution. Larger entities might have more impact than myself.

So I support where we can make a difference with little effort, but a
larger solution is probably out of our reach :-/ Unless of course we
rethink everything...
--
-- dag wieers, dag at wieers.com, http://dag.wieers.com/
-- dagit linux solutions, info at dagit.net, http://dagit.net/

[Any errors in spelling, tact or fact are transmission errors]
Dag Wieers
2011-06-27 13:23:38 UTC
Permalink
Post by Bryan J Smith
You've always thought outside-the-box and too far ahead of a lot of people (and
I say this as the sincerest form of complement).
I've watched the Fedora Project's evolution over many years and while you might
have not been a part of the team or list directly, there's no denying your work
and approaches has indirectly and even directly impacted the project over the
years. I'm not afraid to say that in the least bit if anyone ever asks me, or
makes a statement about DAG, RPMforge, etc...
I mentioned Guidelines, Maintainer and Governance as a kind of "tri-fecta."
Back to my [counter-] analogy, it's more than just a choice (company) detail.
Different locales are trying to solve the same problems, and often use the same
guidelines (industry) and sources (companies) as other locales, but the
maintenance and governance differences end up making all the difference. In
other words, I've never seen it as a technical issue.
I think a big difference is the fact that we always preferred (mostly by
necessity) to have a few people doing lots of packages, rather then a lot
of people doing a few packages. This has the advantage that there's less
overhead to packaging and the process is light and flexible. But the
downside is that we cannot take on large projects, like compatibility with
eg. EPEL.

If you consider ATrpms, rpmfusion, and the myriad of other repositories,
compatibility with other repositories is a hard proposition, and even
tracking changes that may influence compatibility is something a small set
of people cannot undertake. I think there are other and better solutions
if we want the best experience for our users, but this project has more
important concerns than inter-repository compatibility IMO.

Compare it to world-peace or world-hunger, how much I like this to happen,
I am afraid I won't be able to make a big contribution to the overal
solution. Larger entities might have more impact than myself.

So I support where we can make a difference with little effort, but a
larger solution is probably out of our reach :-/ Unless of course we
rethink everything...
--
-- dag wieers, dag at wieers.com, http://dag.wieers.com/
-- dagit linux solutions, info at dagit.net, http://dagit.net/

[Any errors in spelling, tact or fact are transmission errors]
Jeff Johnson
2011-06-27 13:38:33 UTC
Permalink
Disclaimer: I care only for the engineering not the hysterical politics.
Post by Dag Wieers
So there were a few problems that made us decide not to join the Fedora
- They were not interested in RHEL packages
- They were not interested in using macros to simplify supporting
multiple distributions
- They were not interested in using the %dist macro
Let me restate your hysterical interests positively:

- commercially useful "production" quality software distribution.

Enterprise software has long-term maintenance/upgrade/backport issues
that linux distros have never addressed well, but RPMforge (and you) have.
Linux distro packaging policy is typically
Security fixes only.
in order to avoid the non-trivial re-integration costs/risks of
destabilizing already released software product.

- software build recipe's that can be maintained persistently and universally.

By "using macros" you are implicitly stating that a proper abstraction
to hide differences would lead to a unified build recipe that achieves
the ideal holy grail of packaging and software distribution:
Build software anywhere and run binaries everywhere.
Again RPMForge (and you) are unique imho in the approach through a better
methodology, not cost/risk avoidance as in "Security fixes only" and
"Fixed in the next release!" as commonly practiced.

- an identification scheme that can be used for tracking/accountability.

While %dist/%repo identification in Release: strings are rather flimsy (jmho),
there most definitely is a need to identify where/how binaries were built
so that problem reports and feature requests can be delivered accurately.
- the *.spec parser in rpm build was badly borked up in rpm-2.4.12 in 1997

Parsers need a grammar not "Have it your own way!" expectations. The
rpmbuild *.spec parser was tweaked to Get Things Done! and isn't soundly
implemented.

- macros used for templating build recipes are per-distro "de facto" dialects.

While macros/templating are clearly useful (or RPM would have toasted long ago),
the original motivations for the macro implementation were _NOT_ intended
for templating, but rather to unify 3 sources of information passed into a build:
a) per-package build parameters
b) per-build system parameters
c) per-instance command line parameters
What is/was missing (and what you seek imho) is an explicit design to
address the needs of abstracting away and hiding build differences.

- Release: was intended as a simple build++ counter, not for %{?dist} identification.

There are so many sources in need of identification these days that appending
strings to a Release: to indicate how/where software was processed simply
isn't sufficient. The deeper design flaw is that the markers added to
Release: for identification purposes are propagating into "dependency hell"
and breaking upgrades again and again and again for no obvious benefit.

Better engineering *IS* possible to address the flaws I mentioned. ANd it
can be done in a legacy compatible fashion with different templates, not
more macro madness, in order to achieve
Build anywhere and run the binaries everywhere.

The ROADMAP for rpm6.org is currently being devised to address some of
the engineering flaws above. The RPMforge (and your) focus on long
term enterprise focussed and maintainable is well known (to me) and
I fully intend to re-use RPMForge *.src.rpm build recipes as de facto
real world test cases for attempting better solutions to more useful
(than *.spec) build recipes.

hth

73 de Jeff
Nico Kadel-Garcia
2011-06-28 12:40:25 UTC
Permalink
Post by Jeff Johnson
Disclaimer: I care only for the engineering not the hysterical politics.
Post by Dag Wieers
So there were a few problems that made us decide not to join the Fedora
?- They were not interested in RHEL packages
?- They were not interested in using macros to simplify supporting
? ?multiple distributions
?- They were not interested in using the %dist macro
Nothing hysterical about these claims. Dag has been very cooperative,
particularly in encouraging. EPEL has been backporting from Fedora,
which is useful, but the package naming issue remains confusing? The
'dtag' versus 'dist' issue, alone, makes RPMforge easier to integrate
and avoid confusing overlaps with existing RHEL, CentOS, or SL
packages.

Don't get me *started* on the uppercase/lowercase package name thing.
and the perl numbering issues (due to the CPAN standard of using
floating point numbes, which no other significant software repo does
that I can find) Let's focus on an easy one: Nagios plugins. Whoever
submitted that to EPEL elected, understandably, to refactor the
nagios-plugins package and separate out various components. This makes
sense: the dependencies for a plugin like the Quake test are hardly
necessary for most production environments.

RPMforge kep thte nagios-plugins package intact. Chaos ensued when
EPEL went to a higher "release" number, and it wound up replacing my
similarly versioned RPMforge packagte and did not include a whole
stack of monitors.
Post by Jeff Johnson
? - commercially useful "production" quality software distribution.
? ? ? ?Enterprise software has long-term maintenance/upgrade/backport issues
? ? ? ?that linux distros have never addressed well, but RPMforge (and you) have.
? ? ? ?Linux distro packaging policy is typically
? ? ? ? ? ? ? ?Security fixes only.
? ? ? ?in order to avoid the non-trivial re-integration costs/risks of
? ? ? ?destabilizing already released software product.
? ?- software build recipe's that can be maintained persistently and universally.
? ? ? ?By "using macros" you are implicitly stating that a proper abstraction
? ? ? ?to hide differences would lead to a unified build recipe that achieves
Please don't overstate. They've been a very useful tool, indeed for
maintaining consistency, and tracking the differences, by using a well
integated code base.
Post by Jeff Johnson
? ? ? ? ? ? ? ?Build software anywhere and run binaries everywhere.
? ? ? ?Again RPMForge (and you) are unique imho in the approach through a better
? ? ? ?methodology, not cost/risk avoidance as in "Security fixes only" and
? ? ? ?"Fixed in the next release!" as commonly practiced.
Welcome to "the bazaar" of free and open software. I, for one, can't
wait 4 years for the next major release of our favorite upstream
vendor's commercial code base, especially when dealing with system
feature additions (such as perl modules for developers), and such as
Subversion which had glaring performance and security improvements
with version 1.6.x, but which SL 5 was burdened with at version 1.4.x
until the upstream vendor finally gave way after 3 years and updated.
RPMforge had kept up to date, and we all benefitted. (I submitted some
of them of those updates.)
Post by Jeff Johnson
? ?- an identification scheme that can be used for tracking/accountability.
? ? ? ?While %dist/%repo identification in Release: strings are rather flimsy (jmho),
? ? ? ?there most definitely is a need to identify where/how binaries were built
? ? ? ?so that problem reports and feature requests can be delivered accurately.
? ?- the *.spec parser in rpm build was badly borked up in rpm-2.4.12 in 1997
? ? ? ?Parsers need a grammar not "Have it your own way!" expectations. The
? ? ? ?rpmbuild *.spec parser was tweaked to Get Things Done! and isn't soundly
? ? ? ?implemented.
? ?- macros used for templating build recipes are per-distro "de facto" dialects.
? ? ? ?While macros/templating are clearly useful (or RPM would have toasted long ago),
? ? ? ?the original motivations for the macro implementation were _NOT_ intended
? ? ? ? ? ? ? ?a) per-package build parameters
? ? ? ? ? ? ? ?b) per-build system parameters
? ? ? ? ? ? ? ?c) per-instance command line parameters
? ? ? ?What is/was missing (and what you seek imho) is an explicit design to
? ? ? ?address the needs of abstracting away and hiding build differences.
? ?- Release: was intended as a simple build++ counter, not for %{?dist} identification.
? ? ? ?There are so many sources in need of identification these days that appending
? ? ? ?strings to a Release: to indicate how/where software was processed simply
? ? ? ?isn't sufficient. The deeper design flaw is that the markers added to
? ? ? ?Release: for identification purposes are propagating into "dependency hell"
? ? ? ?and breaking upgrades again and again and again for no obvious benefit.
It's a *start*, and it's a small and effective tweak that gets past
the immediate hurdles. Don't let theoretical perfection prevent doing
work that is good enough right now.
Post by Jeff Johnson
Better engineering *IS* possible to address the flaws I mentioned. ANd it
can be done in a legacy compatible fashion with different templates, not
more macro madness, in order to achieve
? ? ? ?Build anywhere and run the binaries everywhere.
And will take 10 years on the current RPM development roadmap, and is
likely to break RPM. Let's not go there.
Post by Jeff Johnson
The ROADMAP for rpm6.org is currently being devised to address some of
the engineering flaws above. The RPMforge (and your) focus on long
term enterprise focussed and maintainable is well known (to me) and
I fully intend to re-use RPMForge *.src.rpm build recipes as de facto
real world test cases for attempting better solutions to more useful
(than *.spec) build recipes.
Any hints of what might be on that roadmap? rpm6.org is a blank Mac
hosted webserver that begs for acceptance of an unauthenticated SSL
key, without a hint of data. SL 6 users are held, for normal
stability, at RPM version 4.6, and rpm.org's last comment on a roadmap
is at version 4.6. Even Fedora 15 is at 4.9. There is a long, long,
long way to go before RPM version 6 might see production use, and most
of us need to deal with existing systems for the next 5 years.

This is a great example of where "good enough" engineering, such as
RPMforge's use of 'dtag', can really help keep things working until
and unless a major rebuild occurs. And doing that rebuild is going to
be *hard*, due to the temendous existing code base.
Nico Kadel-Garcia
2011-06-28 12:40:25 UTC
Permalink
Post by Jeff Johnson
Disclaimer: I care only for the engineering not the hysterical politics.
Post by Dag Wieers
So there were a few problems that made us decide not to join the Fedora
?- They were not interested in RHEL packages
?- They were not interested in using macros to simplify supporting
? ?multiple distributions
?- They were not interested in using the %dist macro
Nothing hysterical about these claims. Dag has been very cooperative,
particularly in encouraging. EPEL has been backporting from Fedora,
which is useful, but the package naming issue remains confusing? The
'dtag' versus 'dist' issue, alone, makes RPMforge easier to integrate
and avoid confusing overlaps with existing RHEL, CentOS, or SL
packages.

Don't get me *started* on the uppercase/lowercase package name thing.
and the perl numbering issues (due to the CPAN standard of using
floating point numbes, which no other significant software repo does
that I can find) Let's focus on an easy one: Nagios plugins. Whoever
submitted that to EPEL elected, understandably, to refactor the
nagios-plugins package and separate out various components. This makes
sense: the dependencies for a plugin like the Quake test are hardly
necessary for most production environments.

RPMforge kep thte nagios-plugins package intact. Chaos ensued when
EPEL went to a higher "release" number, and it wound up replacing my
similarly versioned RPMforge packagte and did not include a whole
stack of monitors.
Post by Jeff Johnson
? - commercially useful "production" quality software distribution.
? ? ? ?Enterprise software has long-term maintenance/upgrade/backport issues
? ? ? ?that linux distros have never addressed well, but RPMforge (and you) have.
? ? ? ?Linux distro packaging policy is typically
? ? ? ? ? ? ? ?Security fixes only.
? ? ? ?in order to avoid the non-trivial re-integration costs/risks of
? ? ? ?destabilizing already released software product.
? ?- software build recipe's that can be maintained persistently and universally.
? ? ? ?By "using macros" you are implicitly stating that a proper abstraction
? ? ? ?to hide differences would lead to a unified build recipe that achieves
Please don't overstate. They've been a very useful tool, indeed for
maintaining consistency, and tracking the differences, by using a well
integated code base.
Post by Jeff Johnson
? ? ? ? ? ? ? ?Build software anywhere and run binaries everywhere.
? ? ? ?Again RPMForge (and you) are unique imho in the approach through a better
? ? ? ?methodology, not cost/risk avoidance as in "Security fixes only" and
? ? ? ?"Fixed in the next release!" as commonly practiced.
Welcome to "the bazaar" of free and open software. I, for one, can't
wait 4 years for the next major release of our favorite upstream
vendor's commercial code base, especially when dealing with system
feature additions (such as perl modules for developers), and such as
Subversion which had glaring performance and security improvements
with version 1.6.x, but which SL 5 was burdened with at version 1.4.x
until the upstream vendor finally gave way after 3 years and updated.
RPMforge had kept up to date, and we all benefitted. (I submitted some
of them of those updates.)
Post by Jeff Johnson
? ?- an identification scheme that can be used for tracking/accountability.
? ? ? ?While %dist/%repo identification in Release: strings are rather flimsy (jmho),
? ? ? ?there most definitely is a need to identify where/how binaries were built
? ? ? ?so that problem reports and feature requests can be delivered accurately.
? ?- the *.spec parser in rpm build was badly borked up in rpm-2.4.12 in 1997
? ? ? ?Parsers need a grammar not "Have it your own way!" expectations. The
? ? ? ?rpmbuild *.spec parser was tweaked to Get Things Done! and isn't soundly
? ? ? ?implemented.
? ?- macros used for templating build recipes are per-distro "de facto" dialects.
? ? ? ?While macros/templating are clearly useful (or RPM would have toasted long ago),
? ? ? ?the original motivations for the macro implementation were _NOT_ intended
? ? ? ? ? ? ? ?a) per-package build parameters
? ? ? ? ? ? ? ?b) per-build system parameters
? ? ? ? ? ? ? ?c) per-instance command line parameters
? ? ? ?What is/was missing (and what you seek imho) is an explicit design to
? ? ? ?address the needs of abstracting away and hiding build differences.
? ?- Release: was intended as a simple build++ counter, not for %{?dist} identification.
? ? ? ?There are so many sources in need of identification these days that appending
? ? ? ?strings to a Release: to indicate how/where software was processed simply
? ? ? ?isn't sufficient. The deeper design flaw is that the markers added to
? ? ? ?Release: for identification purposes are propagating into "dependency hell"
? ? ? ?and breaking upgrades again and again and again for no obvious benefit.
It's a *start*, and it's a small and effective tweak that gets past
the immediate hurdles. Don't let theoretical perfection prevent doing
work that is good enough right now.
Post by Jeff Johnson
Better engineering *IS* possible to address the flaws I mentioned. ANd it
can be done in a legacy compatible fashion with different templates, not
more macro madness, in order to achieve
? ? ? ?Build anywhere and run the binaries everywhere.
And will take 10 years on the current RPM development roadmap, and is
likely to break RPM. Let's not go there.
Post by Jeff Johnson
The ROADMAP for rpm6.org is currently being devised to address some of
the engineering flaws above. The RPMforge (and your) focus on long
term enterprise focussed and maintainable is well known (to me) and
I fully intend to re-use RPMForge *.src.rpm build recipes as de facto
real world test cases for attempting better solutions to more useful
(than *.spec) build recipes.
Any hints of what might be on that roadmap? rpm6.org is a blank Mac
hosted webserver that begs for acceptance of an unauthenticated SSL
key, without a hint of data. SL 6 users are held, for normal
stability, at RPM version 4.6, and rpm.org's last comment on a roadmap
is at version 4.6. Even Fedora 15 is at 4.9. There is a long, long,
long way to go before RPM version 6 might see production use, and most
of us need to deal with existing systems for the next 5 years.

This is a great example of where "good enough" engineering, such as
RPMforge's use of 'dtag', can really help keep things working until
and unless a major rebuild occurs. And doing that rebuild is going to
be *hard*, due to the temendous existing code base.

Bryan J Smith
2011-06-27 13:07:54 UTC
Permalink
You've always thought outside-the-box and too far ahead of a lot of people (and
I say this as the sincerest form of complement).

I've watched the Fedora Project's evolution over many years and while you might
have not been a part of the team or list directly, there's no denying your work
and approaches has indirectly and even directly impacted the project over the
years. I'm not afraid to say that in the least bit if anyone ever asks me, or
makes a statement about DAG, RPMforge, etc...

I mentioned Guidelines, Maintainer and Governance as a kind of "tri-fecta."
Back to my [counter-] analogy, it's more than just a choice (company) detail.
Different locales are trying to solve the same problems, and often use the same
guidelines (industry) and sources (companies) as other locales, but the
maintenance and governance differences end up making all the difference. In
other words, I've never seen it as a technical issue.

Just my observations, as a purposeful outsider to both RPMforge and Fedora/EPEL,
even if annoying to many. ;)




----- Original Message ----
From: Dag Wieers <dag at wieers.com>
Sent: Mon, June 27, 2011 8:27:35 AM

...
So there were a few problems that made us decide not to join the Fedora
project at that time:
- They were not interested in RHEL packages
- They were not interested in using macros to simplify supporting
multiple distributions
- They were not interested in using the %dist macro
...
If you look at today:
- Fedora is doing RHEL packages
- Fedora is using macros to simplify maintaining packages
- Fedora has started using %dist macro in an incompatible way, so we
introduced the %dtag macro for our specific use-case
...
That said, for a large part I agree with the Fedora packaging guidelines
and Tom Callaway has been doing a great job at that. But the Fedora
packaging guidelines are *not* the reason why there are incompatibilities.
Jeff Johnson
2011-06-27 13:38:33 UTC
Permalink
Disclaimer: I care only for the engineering not the hysterical politics.
Post by Dag Wieers
So there were a few problems that made us decide not to join the Fedora
- They were not interested in RHEL packages
- They were not interested in using macros to simplify supporting
multiple distributions
- They were not interested in using the %dist macro
Let me restate your hysterical interests positively:

- commercially useful "production" quality software distribution.

Enterprise software has long-term maintenance/upgrade/backport issues
that linux distros have never addressed well, but RPMforge (and you) have.
Linux distro packaging policy is typically
Security fixes only.
in order to avoid the non-trivial re-integration costs/risks of
destabilizing already released software product.

- software build recipe's that can be maintained persistently and universally.

By "using macros" you are implicitly stating that a proper abstraction
to hide differences would lead to a unified build recipe that achieves
the ideal holy grail of packaging and software distribution:
Build software anywhere and run binaries everywhere.
Again RPMForge (and you) are unique imho in the approach through a better
methodology, not cost/risk avoidance as in "Security fixes only" and
"Fixed in the next release!" as commonly practiced.

- an identification scheme that can be used for tracking/accountability.

While %dist/%repo identification in Release: strings are rather flimsy (jmho),
there most definitely is a need to identify where/how binaries were built
so that problem reports and feature requests can be delivered accurately.
- the *.spec parser in rpm build was badly borked up in rpm-2.4.12 in 1997

Parsers need a grammar not "Have it your own way!" expectations. The
rpmbuild *.spec parser was tweaked to Get Things Done! and isn't soundly
implemented.

- macros used for templating build recipes are per-distro "de facto" dialects.

While macros/templating are clearly useful (or RPM would have toasted long ago),
the original motivations for the macro implementation were _NOT_ intended
for templating, but rather to unify 3 sources of information passed into a build:
a) per-package build parameters
b) per-build system parameters
c) per-instance command line parameters
What is/was missing (and what you seek imho) is an explicit design to
address the needs of abstracting away and hiding build differences.

- Release: was intended as a simple build++ counter, not for %{?dist} identification.

There are so many sources in need of identification these days that appending
strings to a Release: to indicate how/where software was processed simply
isn't sufficient. The deeper design flaw is that the markers added to
Release: for identification purposes are propagating into "dependency hell"
and breaking upgrades again and again and again for no obvious benefit.

Better engineering *IS* possible to address the flaws I mentioned. ANd it
can be done in a legacy compatible fashion with different templates, not
more macro madness, in order to achieve
Build anywhere and run the binaries everywhere.

The ROADMAP for rpm6.org is currently being devised to address some of
the engineering flaws above. The RPMforge (and your) focus on long
term enterprise focussed and maintainable is well known (to me) and
I fully intend to re-use RPMForge *.src.rpm build recipes as de facto
real world test cases for attempting better solutions to more useful
(than *.spec) build recipes.

hth

73 de Jeff
Bryan J Smith
2011-06-26 12:07:10 UTC
Permalink
From: Yury V. Zaytsev <yury at shurup.com>
This question is in a very same vein as asking why would we have 5 major
car platform manufacturers producing mutually incompatible products,
which results in the duplication of the engineering efforts instead of
having one worldwide meta-manufacturer, that would produce the most
advanced car platforms in a truly efficient way.
Do you have an answer for that already?
Maybe it's because I'm a Libertarian-thinking American engineer, but I do not
believe that is an accurate analogy. Yes, I'm nitpicking ... ;)

Just want to point out the following reasons:
1. The assumption that multiple engineering teams are poor and less efficient
2. That a single entity or "bureau" would produce something more advanced
3. Many manufacturers actually use the exact same suppliers, with full
compatibility

If #1 is true, why not just have all engineers in a class work on a single
"super project" instead of challenging each other to see what ideas and designs
they could come up with? And #2, well, that's one of the greatest lessons in
engineering, as the concept of communism actually appeals to engineers, and most
of the Soviet Union was filled engineers in their political landscape. It's the
core belief that everything from marketing to microeconomics is a waste of time,
which is a clear west v. east engineering mentality, whereas the west learns
management and feasibility, and the east did not. And if you don't believe #3,
let's talk the "lowly" Corvette versus Ferrari sometime, and some of the rather
inexpensive, Anglo-American parts that do into high-end products of the latter.
Heck, even the Toyota mats came from an American manufacturer who spends 97% of
their time manufacturing for other makers (and they regularly had to point out
Toyota's requirements and specifications versus others, to show where the
differences came from).

A _better_ analogy would not be the independent, commercial car companies, but
more on the "goverance" side, conflicting North American transportation
standards with European Union, etc..., even inter-country or municipality
issues. Everything from headrests to lights, much more other aspects like
emissions and fuel -- including why Americans are seemingly "allergic" to diesel
(regulatory more than anything). Then you have both the manufacturers as well
as the consumers who have to deal with those conflicting standards, and redesign
even their own vehicles for different locales and regulatory requirements.
That's more of what is going on here, different standards, foci, maintainership,
leadership, etc... GM, Toyota, etc... have wildly differing divisions and even
names (e.g., GM's Holden in Australia), and if you ever have the chance to talk
to a chief engineer, you'll learn a lot about the issues. We can even talk
about the folly of assuming displacement means less fuel efficient, related
taxes and other things.

So does one just accept the Fedora Guidelines, Maintainer and Project governance
as "The" standard? That would certainly solve the problem, correct? How many
here would agree to that? ;)

Let's throw a more simplified auto analogy, racing. Here in the US, for sports
car racing (don't assume just because I'm an American that I follow NASCAR and
we Americans don't have anything but oval tracks ;) ), there is both IMSA
sponsored, ACO (EU) aligned, series like the American Le Mans Series (ALMS), and
more US-centric ISC run Grand Am program. Many would argue that the ACO is
already a real, international organization, with standards for prototype and
touring that everyone should follow. Others would complain there are some
serious differences in foci for racing between NA and EU, especially when it
comes to manfacturing-centric v. performance-centric. Both arguments have
merits. As much as the ACO has had quite a history, one can look back to the
'80s and Grand Am's predecessor in influencing major advances in everything from
ABS to traction control.


And that's before we consider the US' manufacturing-centric industry has
generally been allergic to formal sanction for fear of racing being seen as
dangerous and a liability for sales. Even GM was a major opponent (until
1998), and Ford only officially sanctioned it's '60s ACO teams because it was
spurned when it tried to by Ferrari (and even then it did not use its own
program, but looked to Shelby who looked outside of Ford). And even when
sanctioned today by the ACO, you have differences between NA and EU. ALMS is in
the bread'n butter consumer markets of the Americas, so it's clearly more
manufacturing-based. Even LMS (EU) is far more centric on the privateer teams
and drivers -- especially in FIA GT1 where few manufacturers go. I mean, if
you're a smaller design firm, who wants to challenge a deep pockets corporation
like a GM or Toyota when they have product branded manufacturer backed teams
that sell sub-$50K models that beat your 6-7 figure priced model continually?
Does your marketing absolutely no good.

Governance, not choice, is the issue. And playing fields are rarely level.
Different requirements tend to cause different needs. However, focusing on
co-existence and compatibility is a worthwhile endeavor, even if the governance
is not shared.
--
Bryan J Smith Professional, Technical Annoyance
Linked Profile: http://www.linkedin.com/in/bjsmith
Steve Huff
2011-06-26 12:20:28 UTC
Permalink
Post by Bryan J Smith
Bryan J Smith Professional, Technical Annoyance
yeah, i remember you from the CentOS list. if you would, please post lengthy and tangential content like this to a blog. thanks!

-steve
--
If this were played upon a stage now, I could condemn it as an improbable fiction. - Fabian, Twelfth Night, III,v
PGP 8477B706 (A92A 1F7E 6D76 16A0 BFF9 E61D AD54 0251 8477 B706)
Yury V. Zaytsev
2011-06-26 17:46:23 UTC
Permalink
Post by Bryan J Smith
Maybe it's because I'm a Libertarian-thinking American engineer, but I do not
believe that is an accurate analogy. Yes, I'm nitpicking ... ;)
I didn't mean to construct an accurate analogy. I wanted to point out
that "divergent development" exists just about everywhere outside of the
packaging world and there isn't always a way or even the need to fix
it.
Post by Bryan J Smith
However, focusing on co-existence and compatibility is a worthwhile
endeavor, even if the governance is not shared.
I think I was very explicit in my message, that packagers both behind
RepoForge and EPEL are interested in staying interoperable.
Post by Bryan J Smith
I've been a lurker for years on this list, and I assist others off-list and
privately precisely because individual like yourself enjoy questioning me
on-list. Learned that years from the CentOS list. ;)
It seems to me that somehow you've got it all wrong: assisting others
on-list is welcome and in fact is the purpose of this list, whereas
verbose off-topic discussions are not.

Therefore, I'd also appreciate the follow-up conversations on this topic
would taking place where they are more appropriate.
--
Sincerely yours,
Yury V. Zaytsev
Dag Wieers
2011-06-27 12:27:35 UTC
Permalink
Post by Bryan J Smith
From: Yury V. Zaytsev <yury at shurup.com>
This question is in a very same vein as asking why would we have 5 major
car platform manufacturers producing mutually incompatible products,
which results in the duplication of the engineering efforts instead of
having one worldwide meta-manufacturer, that would produce the most
advanced car platforms in a truly efficient way.
Do you have an answer for that already?
So does one just accept the Fedora Guidelines, Maintainer and Project governance
as "The" standard? That would certainly solve the problem, correct? How many
here would agree to that? ;)
Let me state again that when the Fedora project started off, RPMforge (or
rather my repository) already existed. I was very interested to join the
Fedora project on the premise that they would be doing RHEL packages as
well. Matthias (also part of the RPMforge project) did join the Fedora
project because he was mostly interested in Fedora packages anyway.

So there were a few problems that made us decide not to join the Fedora
project at that time:

- They were not interested in RHEL packages

- They were not interested in using macros to simplify supporting
multiple distributions

- They were not interested in using the %dist macro

You can find the heated discussions in a Fedora archive somewhere. The
reasoning was that they only had to provide packages for the latest Fedora
and any backward-compatibility would be more easily handled by forking
SPEC files.

While obviously we were looking at keeping packages updated for 3 to 4
distributions (RHEL2, RHEL3, RHEL4 and RHEL5) over a 7 year time-span. So
forking meant in the worst case making the same update to 4 SPEC files.

If you look at today:

- Fedora is doing RHEL packages

- Fedora is using macros to simplify maintaining packages

- Fedora has started using %dist macro in an incompatible way, so we
introduced the %dtag macro for our specific use-case

So I think it is a shame that the few voices from RPMforge got lost in the
very heated discussions of the crowd and at a certain point I simply quit
the list because you simply cannot argue a crowd that is unable to look at
things from a different perspective.

Sadly, the above led to the differences and incompatibilities we have
today. And I feel that there's nothing in my power that I could have done
differently to avoid it. And I have been very angry when Fedora started
doing EPEL and started introducing problems without tagging their packages
so they were easily identifiable as being EPEL's.

That said, for a large part I agree with the Fedora packaging guidelines
and Tom Callaway has been doing a great job at that. But the Fedora
packaging guidelines are *not* the reason why there are incompatibilities.
--
-- dag wieers, dag at wieers.com, http://dag.wieers.com/
-- dagit linux solutions, info at dagit.net, http://dagit.net/

[Any errors in spelling, tact or fact are transmission errors]
Loading...