|
|
|
|
|
|
|
| |
|
|||||
| |
|
|
|
|
|
|
| |
||||||
 |
 |
 |
 |
 |
 |
Installing & Upgrading Instructions
To install and set up DNews simply follow the installation instructions below. Apart from the site-specific information you will enter during installation, DNews is virtually ready to go.
Example: dnewsXXX.exe
Notes:
Windows NT Upgrading
Notes for Windows 95/98 Installation Follow the instructions for Windows NT and mostly it will work except for:
out_size 10240
body_chunk 10240
max_out 400000
max_out_total 400000
UNIX Installation (or upgrading)
/home/chris>
uncompress dnews43a_linux.tar
/home/chris> tar -xvf dnews43a_linux.tar
/home/chrisp> cd dtemp
/home/chrisp/dtemp> su root
/home/chrisp/dtemp> ./dnews_setup
Skip to step 10 below, or read thru the 'manual' installation instructions if you want to know what the setup utility has done for you:
EDIT dnews.conf
Modify the file DNEWS.CONF to describe your system by defining the following symbols
| SYMBOL | Value | Description |
|---|---|---|
| manager | system | email address or user@site.com |
| nntp_feeder | news.upstream.site | Place to suck news from myname news.your.domain Name of your server |
| df | /usr/bin/df -k | path and options for df command to get posix standard output, df -Pk sometimes works |
| diskspace | 100 | space available in megabytes 1,000,000 bytes (not 1024^2) |
| sendmail | /usr/bin/sendmail | The path and flags for sendmail or equiv. |
| confirm | *your.domain | Enable email confirmation when item is passed up to a feeder site. |
On LINUX /etc/rc.d/rc.local add a line:
/usr/local/dnews/dnews_start.sh
On some systems this is not needed as the run level startup (/etc/rc.d/init.d) will be used via dnews.init, be sure you don't get two DNews processes installed :-)
*.this.place:read,post:::*
161.23.4.*:read,post:::*
su root (You need priv to install the files start
DNews)
./install.sh
cd /usr/local/dnews
./dnews_start.sh
The system will now try and get a list of news groups from your feeder site. While this is happening it will not respond, but you can check its progress by:
more dnews.log
When it has finished (give it 10-20 minutes), you can then try:
tellnews status
Note: With the above setup DNews will automatically suck (pull) news from your provider. To set-up DNews to receive a traditional full IHAVE feed you should also refer to 'Instructions on setting up DNews to take a full IHAVE feed' in the Feeds In section of this manual.
Swap partition:
NEWS is memory hungry, you don't need a lot of physical memory but you do need a large swap partition. As a rule of thumb 100-200MB is a good idea for 3-10Gig of news, some brands of unix (freebsd and bsdi) seem to need more than others.
LINUX file handles:
If you expect to have more than 125 concurrent users then you will need to modify your kernal to allow more file handles. On linux you would modify these files in the /usr/src/linux... directory.
in "fs.h" NR_OPEN 1024 in "limits.h" NR_OPEN 1024 OPEN_MAX 1024 in "posix_types.h" __FD_SETSIZE 1024
Then rebuild the kernal in the normal way (see /usr/src/linux/README)
Also you will need a recent linux5 build of DNews, e.g. dnews47d_linux5.tar.Z
Comes with a dummy nntp daemon installed, you must remove this so DNews can bind to the socket. (so it won't even start up)
edit /etc/inetd.conf (comment out the NNTP line)
kill -HUP
OSF (Digital Unix)
To disable the unaligned access warnings add this line to dnews_start.sh, these unaligned accesses do not affect performance. (or upgrade to DNews 4.2i or later)
uac p 0
Startup scripts are sometimes a problem, they tend to be different on every version of unix and between versions of the same unix system.
This command may be helpful if you are trying to use the init script :-)
chkconfig --add dnews.init
If you want to use DNews on HP-UX you have to customize the kernal: Per default a process is only able to allocate 60MB. (This might be not enough for DNews) You have to set the kernal parameter MAXDSIZ to a higher value. (It is easy to do this using SAM - otherwise you have to take care: the value is in hex (not in dec) and in pages (not in bytes))
We recommend you increase the limit to about 220MB so it will never run out.
If you need to run with more than 125 concurrent users then you should probably rebuild your kernel with maxusers set to a larger number like 512, failing that these settings below should be considered minimum values:
Add these lines to /etc/rc.local (example minimum values)
sysctl -w kern.maxfiles=8192
sysctl -w kern.maxfilesperprocess=8192
On a large system you may need to build a new kernal to increase the memory process
limits, here is an example:
/usr/src/sys/i386/conf/YOURKERNELSETTINGSFILE
options "MAXDSIZ=(256*1024*1024)"
options "DFLDSIZ=(256*1024*1024)"
To repeat the above settings may not be needed if you build your kernel for a large number of users like 512 then the defaults should be correct for the other settings mentioned here.
You may need to add ulimit -n 1024 to dnews_start.sh and dmulti_start.sh on some versions of solaris. This is only needed if your default limit is below 300 which is a little unusual.
Some settings that 'might' improve solaris pefromance with DNews, these are just things I've come across but not actually tested :-)
In /etc/system
set ufs:ufs_HW=6291456 set ufs:ufs_LW=4194304 set priority_paging=1
(then reboot)
Use dnews5.4f12 or later, and it probably isn't critical but we recommend installing the latest patch for the solaris threading libraries, e.g. http://access1.sun.com/patch.public/cgi-bin/readme2html.cgi?patch=105568
You MUST put this in your /etc/system file: add to FAQ please.
set rlim_fd_max=4096
set rlim_fd_cur=2048
*
set maxusers=2048
BSDI, Unixware, Sunos, FreeBSD
Some brands of unix (BSDI, Unixware, Sunos) have process limits, the two that will hurt DNews are memory use/datasize, and openfiles. The following commands can remove these limits - you will need to do this for root and news users.
limit memoryuse unlimited
limit openfiles 256
limit datasize unlimited
(Bash Sunos increase file handles) ulimit -n 256
With some operating systems BSDI, HPUX, AIX, you also need to increase the KERNAL memory limits, for BSDI, FREEBSD this is done with the following commands, these are permanent settings that remain set even between reboots:
sysctl -w vm.maxssize=90000000
sysctl -w vm.maxdsize=201326592
or
sysctl -w vm.maxdatasize=201326592
You may need to recompile the kernal as well, here are the settings you need to change for BSDI:
Add these options to your config file (/sys/i386/conf/FILENAME) :
# Memory Options
options "DFLDSIZ=\(128*1024*1024\)"
options "MAXDSIZ=\(256*1024*1024\)" # max size of a process
options CHILD_MAX=512 # max live children per parent
options OPEN_MAX=256 # max number of open files
options NMBCLUSTERS=2048 # raises mbuf clusters (increases the
# amount of virtual memory for network
# buffers)
For most other systems these kernal parameters need to be set in the kernal configuration files before or during the process of building a new kernal.
Here is an example of dnews_start.sh for BSDI
#!/bin/sh
cd /usr/local/dnews
ls -ald /usr/local/dnews
chown news /usr/local/dnews/*
chown news /usr/local/dnews
rm nohup.old
mv nohup.out nohup.old
limit datasize unlimited
limit openfiles 256
limit memoryuse unlimited
nohup ./dnews_start &
For MAC OS/X add near the top:
...
ulimit -s 60000
ulimit -d 200000
...
./dmgrsvc
./dnews
...
Here is an example of dnews_start.sh for Unixware (2.03)
#!/bin/sh
cd /usr/local/dnews
ls -ald /usr/local/dnews
chown news /usr/local/dnews/*
chown news /usr/local/dnews
rm nohup.old
mv nohup.out nohup.old
ulimit -n 256
ulimit -d 60000
ulimit -v 60000
nohup ./dnews_start &
To test these limits are really removed, use the commands:
tellnews test_mem
tellnews test_files
You must stop and restart DNews after running these tests. You should see about 200 free file handles and about 30MB or more of memory that can be allocated, if you see less then you haven't removed the limits.
UNIX Upgrading from INN or CNEWS
First, follow the above unix installation instructions, but don't start DNews.
You can copy the following files from your UNIX system:
active --> /var/spool/dnews/work/active.dat
active.times --> /var/spool/dnews/work/active.times
newsgroups --> /var/spool/dnews/work/active.names
Then start DNews, if you want to run both DNews and INN at the same time while testing you can even do this, define port in dnews.conf to some port other than 119. (e.g. 1119)
Now if you have enough disk space (2x spool area) then you can import the spool area into DNews, and run it in parallel until you are happy.
tellnews spool_import /var/spool/news
If you don't have enough disk space to do this, you might just convert a small part of the spool area for testing, e.g.:
tellnews spool_import /var/spool/news/comp/lang
If you then decide you are happy with DNews and you want to convert the entire spool area, deleting the INN files as they are converted (to create enough space) then you can use the command:
tellnews spool_convert /var/spool/news
If you do not mind whether DNews uses exactly the same item numbers as the INN system then you can add this setting to dnews.conf
convert_add true
| SYMBOL | EXAMPLE |
|---|---|
| manager | system (email address or user@site.com) |
| nntp_feeder | news.feeder.site |
| myname | news.this.place |
| timezone | nzst (new zealand standard time) |
| diskspace | 100 (space available in megabytes) |
| smtp (the foreign email protocol name) | |
| confirm | *.this.place Enable email confirmation when item is passed up to a feeder site. |
*.this.place:read,post:::*
(by name)
161.23.4.*:read,post:::* (by number)
$ @dnews_login (to define the symbol tellnews)
The system will now try and get a list of news groups from your feeder site. While this is happening it will not respond, but you can check its progress by:
$ type dnews.log
When it has finished (give it 10-20 minutes), you can then try:
$ tellnews status
Go to a PC and try out your news reader software pointed at the new system.
With the above setup DNews will automatically suck (pull) news from your provider. To set-up DNews to receive a traditional full IHAVE feed you should also refer to 'Instructions on setting up DNews to take a full IHAVE feed' in the Feeds In section of this manual.
WARNING: DO NOT USE ON NETWARE 5.x
This version of DNews DOES NOT WORK on netware 5.x, we may build a netware 5 version but when we do, this warning will be removed so if you are reading this don't use DNews on netware 5. (It almost runs but gets into a cpu loop due to some change in the directory scanning functions between netware 4 and netware 5.)
WARNING: MEMORY
DNews expects virtual memory, as netware has none you have to make sure your system has sufficient REAL memory. For a sucking feed you will need 8MB free, for a full IHAVE feed you will need 16MB-60MB of free memory (these are estimates).
WARNING: GROUPS
This release for NETWARE is limited to 10,000 groups - this is plenty if you take care with the 'ME' setting in newsfeeds.conf
WARNING: BETA RELEASE
This is a beta release, NetWin do not recommend that you install this on a mission
critical server. Please report any problems you find.
attach myserver
File, Run, dnewsnw23f.exe (or whatever the latest version is)
Answer the configuration questions.
In your autoexec.ncf script add the following lines.
load netdbattach myserver
load DNews attach myserver
Load the server by hand.
load netdb
load DNews
Go to a PC and try out your news reader software pointed at the new system. On windows 95
you can run the DNEWSMAN configuration utility to setup or modify your configuration.
Use the command:
load tellnews status To see what it's doing
load tellnews register To see what it's doing
load tellnews exit To shutdown DNews nicely
load tellnews help For other commands.
NEVER shutdown the service with the 'unload' command. Always use load tellnews exit instead.
With the above setup DNews will automatically suck (pull) news from your provider. To set-up DNews to receive a traditional full IHAVE feed you should also refer to 'Instructions on setting up DNews to take a full IHAVE feed' in the Feeds In section of this manual.
pkunzip c:\dtemp\dnews2.XX_os2.zip
e install.cmd
set config=d:\dnews
Then run the installation:
install
e dnews.conf
Modify the file DNEWS.CONF to describe your system by defining the following symbols
| SYMBOL | EXAMPLE |
|---|---|
| manager | youremail@your.site |
| nntp_feeder | news.upstream.site |
| myname | this.machine.name |
| diskspace | 100 Space available in megabytes. |
| Confirm | *.this.place Enable email confirmation when item is passed up to a feeder site. |
- Double click on OS/2 System
- Double click on Startup
- Right click on any object
- Left click on Create Another
- Change 'New Name:' to DNews NNTP Server
- Click on Create
- Set Path to: c:/dnews/dnews.exe
- Set Work Directory: c:/dnews
- Click on Session
- Click on Start Minimized
- Close the dialogue
e access.conf
*.this.place:read,post:::* (by name)
161.23.4.*:read,post:::* (or by number)
Check
the log file and status to see if its running:
> cd \dnews
> dtail dnews.log
> tellnews status
Now try reading news, it may take a minute or two to fetch the list of groups from your server.
With the above setup DNews will automatically suck (pull) news from your provider. To set-up DNews to receive a traditional full IHAVE feed you should also refer to 'Instructions on setting up DNews to take a full IHAVE feed' in the Feeds In section of this manual.
We now only support the Mac/OSX platform, for which you should follow the unix installation instructions above.
Sorry if this has caused confusion, we were not happy with our Mac version and since we could not bring it up to a standard we were happy with we felt it was necessary to withdraw it.
Are you going to take a full traditional news feed including binaries?
If so then you will need about 1-2 Gig per day of news you want to store. (This is an estimate - the volume doubles every 6-12 months.)
If you have less than 4000 users, or if your users are very similar (e.g. your users are all from one company and have similar interests) then sucking news may be a good option.
If you plan on more than 100 concurrent users, then using DMULTI will be essential to give good performance. I recommend it above about 50 concurrent users.
For a full feed system:
spool size = days_of_news * 1.5 Gig
ram = spool_gigs * 5 + 32MB + 200K*concurrent_users
So let's take an example of a full feed, with approx 10 days of news and 100 concurrent users.
Spool_size = 15 Gig
ram = 75 + 32 + .2*100 = 32 + 75 + 20 = 120MB
Processor: Pentium 100 or better.
If you exclude binary groups then the spool would be 7-200Gig.
If you suck news and exclude binaries, then 2-8 gig would be heaps and you could run it on 64MB of RAM. Adding memory is the cheapest upgrade available, and extra 256mb of ram will really improve usenet server performance :-).
See this page for more details on sizing a large system.
This is a rough guide to calculating how much memory DNews will use. This is VIRTUAL memory, you don't need this much physical memory (but it will go faster if it does :-)
(Concurrent connections) * 200K
(Articles in history file - see expire processed) * 12
(Spool Gig's) * 1MB
(Max article size accepted) * 2
(Number of news groups) * 100
head_cache (default 200) * 3k
xover_cache (default 3000) * 300
Program static data 4mb
So let's take a smallish system with 20,000 news groups, 10 concurrent connections, 200,000 lines of history data, 1 Gig of spool, 1MB articles.
10 * 200K = 2MB
200,000 * 12 = 2.4MB
1Gig = 1MB
1MB Articles = 2MB
20,000 News groups, = 2MB
head_cache (default 200) * 3k = 600K
xover_cache (default 3000) * 300 = 900K
Program static data = 4mb
Total memory use = 14.9MB
NOTE: You DO NOT NEED THIS MUCH RAM. DNews is designed to deal with multiple full feeds and hundreds of users. If you are running a smaller system you don't need all this as REAL RAM, that's what virtual memory is for, it just slows down a little.
Typically on a small system where you wish to reduce the memory use you need to look at:
Total groups
History remember setting.
Maximum article size
You can drastically reduce memory use with these settings:
(in expire.conf) remember 4
(in dnews.conf) item_max 200000 (stops large binary items!!!)
(in newsfeeds.conf change your ME feed to only allow groups that really exist in your
country-area)
site ME
groups !*,rec.*,comp.*,alt.*,news.*,soc.*,bit.*,bionet.*,sci.*
tellnews matchfeed
tellnews purgegroups
(restart DNews to free up the memory)
As a rule of thumb to avoid calculations with unix and NT, create a 100MB swap/page file as a first step. If you are running a big system, then 200mb would be a wise choice.