software/!RELEASES/ircservices/achurch.org/services/lists/ircservices/2000/000847.html

   1 <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
   2 <HTML>
   3  <HEAD>
   4    <TITLE> [IRCServices] cheers. ideas.
   5    </TITLE>
   6    <LINK REL="Index" HREF="index.html" >
   7    <LINK REL="made" HREF="mailto:ircservices%40ircservices.za.net?Subject=%5BIRCServices%5D%20cheers.%20ideas.&In-Reply-To=E13jVsB-000CAP-00%40manx.dreamhaven.net">
   8    <META NAME="robots" CONTENT="index,nofollow">
   9    <META http-equiv="Content-Type" content="text/html; charset=us-ascii">
  10    <LINK REL="Previous"  HREF="000846.html">
  11    <LINK REL="Next"  HREF="000848.html">
  12  </HEAD>
  13  <BODY BGCOLOR="#ffffff">
  14    <H1>[IRCServices] cheers. ideas.</H1>
  15     <B>Jonathan Morton</B>
  16     <A HREF="mailto:ircservices%40ircservices.za.net?Subject=%5BIRCServices%5D%20cheers.%20ideas.&In-Reply-To=E13jVsB-000CAP-00%40manx.dreamhaven.net"
  17        TITLE="[IRCServices] cheers. ideas.">chromi at cyberspace.org
  18        </A><BR>
  19     <I>Wed Oct 11 17:36:02 PDT 2000</I>
  20     <P><UL>
  21         <LI>Previous message: <A HREF="000846.html">[IRCServices] cheers. ideas.
  22 </A></li>
  23         <LI>Next message: <A HREF="000848.html">[IRCServices] cheers. ideas.
  24 </A></li>
  25          <LI> <B>Messages sorted by:</B>
  26               <a href="date.html#847">[ date ]</a>
  27               <a href="thread.html#847">[ thread ]</a>
  28               <a href="subject.html#847">[ subject ]</a>
  29               <a href="author.html#847">[ author ]</a>
  30          </LI>
  31        </UL>
  32     <HR>
  33 <!--beginarticle-->
  34 <PRE>&gt;<i>     Okay, my two cents:
  35 </I>&gt;<i>
  36 </I>&gt;<i>     I'll be the first to admit that the current Services database format
  37 </I>&gt;<i>is less than ideal.  When I originally designed Services, I designed it
  38 </I>&gt;<i>the network I had started a bit earlier, which at the time had at most
  39 </I>&gt;<i>20-30 simultaneous users.  Services can write a database of a few thousand
  40 </I>&gt;<i>nicks quickly enough not to matter, but up that by an order of magnitude
  41 </I>&gt;<i>and you'll start seeing lag just from writing the database.  Moreover, a
  42 </I>&gt;<i>single bit error in the wrong place and you lose the entire database (and
  43 </I>&gt;<i>apologies to anyone who's experienced this).
  44 </I>&gt;<i>
  45 </I>&gt;<i>     There's also the issue of accessing, and particularly altering, the
  46 </I>&gt;<i>databases from outside of Services.  Although access alone isn't really a
  47 </I>&gt;<i>problem--if you want to access the Services databases now, you can do it
  48 </I>&gt;<i>by borrowing the database loading code; listchans and listnicks work like
  49 </I>&gt;<i>that--altering the databases from outside of Services' control brings in
  50 </I>&gt;<i>all the classic problems of multithreaded applications--races, deadlocks,
  51 </I>&gt;<i>that sort of stuff.  Which isn't to say it can't be done, of course, but
  52 </I>&gt;<i>as Andrew Kempe commented, it'll take a good deal of work.  My personal
  53 </I>&gt;<i>thought is that it would be better done as a multi-step process: first
  54 </I>&gt;<i>implement some manner of sending Services certain commands from outside
  55 </I>&gt;<i>(e.g. register/drop a nick), iron out the problems there, then look at
  56 </I>&gt;<i>going to complete external read/write support.
  57 </I>
  58 &lt;snip&gt;
  59
  60 OK, so there's two separate issues.  One: the reliability/performance of
  61 the database file format (this *is* an issue, ChatCircuit experienced a
  62 severe db corruption around 29th Feb / 1st March).  Two: the possibility of
  63 external applications wishing to read/update Services data.
  64
  65 In terms of file format, I wonder if it's really necessary to have it all
  66 in one file?  I've heard of many applications which need to handle lots of
  67 data which use many files for storing it, in a systematic manner.  Squid
  68 (the webcache) uses one or two files per cached object, and spreads the
  69 files over a large number of directories in a single tree (this is an
  70 extreme example).  I see no fundamental reason why there shouldn't be a
  71 limit on the number of nicks per database file, and as many database files
  72 as necessary.  A similar scheme for channels could exist, particularly if
  73 there were one file per channel.
  74
  75 Thus corruption to the database is much more limited given corruption of a
  76 single file, and Services' performance while writing files can be optimised
  77 a *lot* because it only needs to write to files that have actually changed.
  78 Also, since less data is being written at one time, it becomes feasible to
  79 write the files in a more &quot;robust&quot; format, including one based on ASCII
  80 such as XML.  Whether this is still a good idea is an entirely separate
  81 issue which I won't try and get involved in.
  82
  83 In terms of external access, what kind of external access do we need?
  84 Read-only?  Writable?  If we only need read-only access, then we can write
  85 a daemon (or Apache module) which is capable of reading database file(s)
  86 directly and interpreting them according to the required application.
  87 Since it would be separate from Services, it shouldn't interfere with
  88 performance too much, especially given an SMP box.  Writable access is more
  89 of a problem, and requires essentially that Services get back in on the
  90 act.  Opening another socket and allowing commands/responses to
  91 retrieve/set data shouldn't be all that much of a problem, IMHO.  Whether
  92 one makes that compatible with any existing protocol is left as a
  93 discussion exercise for the reader.  :)
  94
  95 --------------------------------------------------------------
  96 from:     Jonathan &quot;Chromatix&quot; Morton
  97 mail:     <A HREF="http://www.ircservices.za.net/mailman/listinfo/ircservices">chromi at cyberspace.org</A>  (not for attachments)
  98 uni-mail: <A HREF="http://www.ircservices.za.net/mailman/listinfo/ircservices">j.d.morton at lancaster.ac.uk</A>
  99
 100 The key to knowledge is not to rely on people to teach you it.
 101
 102 Get VNC Server for Macintosh from <A HREF="http://chromatix.autistics.org/vnc/">http://chromatix.autistics.org/vnc/</A>
 103
 104 -----BEGIN GEEK CODE BLOCK-----
 105 Version 3.12
 106 GCS$/E/S dpu(!) s:- a19 C+++ UL++ P L+++ E W+ N- o? K? w--- O-- M++$ V? PS
 107 PE- Y+ PGP++ t- 5- X- R !tv b++ DI+++ D G e+ h+ r- y+
 108 -----END GEEK CODE BLOCK-----
 109
 110
 111
 112 ---------------------------------------------------------------
 113 To unsubscribe, send email to <A HREF="http://www.ircservices.za.net/mailman/listinfo/ircservices">majordomo at ender.shadowfire.org</A>
 114 with &quot;unsubscribe ircservices&quot; in the body, without the quotes.
 115
 116
 117 </PRE>
 118
 119 <!--endarticle-->
 120     <HR>
 121     <P><UL>
 122         <!--threads-->
 123         <LI>Previous message: <A HREF="000846.html">[IRCServices] cheers. ideas.
 124 </A></li>
 125         <LI>Next message: <A HREF="000848.html">[IRCServices] cheers. ideas.
 126 </A></li>
 127          <LI> <B>Messages sorted by:</B>
 128               <a href="date.html#847">[ date ]</a>
 129               <a href="thread.html#847">[ thread ]</a>
 130               <a href="subject.html#847">[ subject ]</a>
 131               <a href="author.html#847">[ author ]</a>
 132          </LI>
 133        </UL>
 134
 135 </body></html>