2004-7-8
Jeremy Bowers: Re: [Pyds-dev] Re: [PyCS-devel] Re: Maybe problems with Metakit 2.4.9.3
I've poked around a bit more and another possibility is ZODB. It can be
used either as a multi-process database or a multi-threaded single file.
I don't know the internals of PyCS, but it would require a couple of
changes in PyDS. The biggest change is that ZODB stores Python objects,
not just class-free database rows. Overall, I think this would be a good
change anyhow.
It looks to me like you'd need to replace current Metakit functionality
with an object that would probably sub-class a BTree
(http://zope.org/Wikis/ZODB/FrontPage/guide/node6.html#SECTION000630000000000000000)
and also index by other interesting attributes, probably with a simple
dict (at least for PyDS, PyCS may need other BTrees).
http://zope.org/Wikis/ZODB/FrontPage/guide/index.html
http://www.python.org/workshops/2000-01/proceedings/papers/fulton/zodb3.html
Comment on this post [ so far]
Phillip Pearson: Re: [Pyds-dev] Re: [PyCS-devel] Re: Maybe problems with Metakit 2.4.9.3
> >I'm seriously considering moving PyCS over to MySQL or Postgres
>
> As long as you write a migration script that's fine. And as long as you
> use PostgreSQL or write it database independend, as I currently don't
> run MySQL on simon.bofh.ms ;-)
Yeah - Postgres was my first choice anyway :)
The migration script will be the first thing I do ... I have a server
to transition too!
Cheers,
Phil :)
Comment on this post [ so far]
Louis Feges: RE: [Pyds-dev] Re: [PyCS-devel] Re: Maybe problems with Metakit 2.4.9.3
>Another problem with SQLite is, it only handles one writer while
>allowing multiple readers. That's the same as with Metakit (ok, you can
>have multiple database connections with one writer each, but that will
>end in file locking if I understand it's architecture correctly).
As far as I know, SQLite cannot handle multiple "writer" connections. My
understanding is that it is a single writer, supporting multiple readers; as
is metakit, I presume. SQLite has a small footprint, supports standard SQL,
and is easily extensible via C or C++. Ideal for embedded systems, personal
dbms, single user applications. I think it is the open source equivalent of
Microsoft's MSAccess without the builtin forms, reports, etc.
It has all the features of a robust database such as transaction
commit/rollback, lock resolution, etc.; but is not intended as a
transactional, multi-user dbms. That's where MySQL and Postgress come in.
For PyDS, I would suggest implementing with SQLite, initially, and building
on a port that supports other sql databases, such as is done by Roundup
http://roundup.sourceforge.net/doc-0.7/features.html, an issue management
solution written in Python, supporting multiple database implementations.
\louis
Comment on this post [ so far]
Georg Bauer: Re: [Pyds-dev] Re: [PyCS-devel] Re: Maybe problems with Metakit 2.4.9.3
Hi!
> If you're porting databases for PyDS, I'd love to see `DB-API2`_ used
> so we
> can put PyDS on any [relational] database we need to. If you want a
> more
> Pythonic abstraction to work with, SQLObject_ is nice. I must admit,
> I've
For PyDS that actually would be the way to go, as SQLObject gives me
some abstraction in the way I have now with Metakit. I don't know
wether it's really the right way for PyCS, though. I think it should
directly use DBAPI and I think it should go to a full blown SQL server,
not SQLite (although that would stay an option because of DBAPI)
because of the higher load on those systems. Both muensterland.org and
pycs.net get quite a big whip of traffic every day and I think a
separate SQL server is in order there.
Another problem with SQLite is, it only handles one writer while
allowing multiple readers. That's the same as with Metakit (ok, you can
have multiple database connections with one writer each, but that will
end in file locking if I understand it's architecture correctly). One
thing that is problematic with the current PyCS is it's blocking
nature, as all extensions are run in the main thread. Maybe it would be
a good idea to port over PyCS modules to TooFPy to get multiple thread
handlers for dynamic stuff (you keep the single threaded
highperformance medusa engine for static stuff with TooFPy!).
bye, Georg
Comment on this post [ so far]
Georg Bauer: Re: [Pyds-dev] Re: [PyCS-devel] Re: Maybe problems with Metakit 2.4.9.3
Hi!
> (I would imagine PyCS would still want MySQL or Postgres, truthfully,
> but it is worth a look. PyDS might be a better fit with SQLite. I
> *like* being able to backup my PyDS just by shutting it down and
> copying some files. I think SQLite should retain that, it isn't so
> easy for MySQL.)
Yes, PyDS is a different beast. I don't think I will use a full blown
SQL database for that one. I am not too happy with SQLite, too, as it's
SQL is sometimes rather weird, but since rewriting PyDS to another
database is definitely a bigger project, I am a bit reluctant to start
that :-)
So PyDS stays with Metakit for now, but I think I will change in the
long run. I am just not sure which way to go and when to do it.
bye, Georg
Comment on this post [ so far]
Georg Bauer: Re: [Pyds-dev] Re: [PyCS-devel] Re: Maybe problems with Metakit 2.4.9.3
Hi!
> I'm seriously considering moving PyCS over to MySQL or Postgres
> (probably the latter, b/c the license fits better). Georg -
> would you mind if I did something like that?
As long as you write a migration script that's fine. And as long as you
use PostgreSQL or write it database independend, as I currently don't
run MySQL on simon.bofh.ms ;-)
Actually I am not that happy with Metakit, too. I think it is
responsible for some weird lockups/segfaults I have from time to time
and we do have reports of data corruption from users.
bye, Georg
Comment on this post [ so far]