Quick FTP Setup

This is a quick and dirty setup guide to enabling ftp uploads for your server to take advantage of FTP file transfer speeds which are much faster than SFTP/SCP. It only applies to servers in a secured lab environment, and should never be performed on production servers or public-facing servers.

- check whether vsftpd is installed:

rpm -q vsftpd

yum install vsftpd

- configure it:

chmod 777 /var/ftp/pub

vi /etc/vsftpd/vsftpd.conf

change anon_upload_enable and anon_mkdir_write_enable to YES; add line with: ftp_username=nobody (so that will prompt for user in RedHat linux)

service vsftpd start

ftp login using user anonymous with blank password



Note:

you may need to disable iptables and selinux.

refs:

http://nixcraft.com/getting-started-tutorials/725-secure-vsftpd-ftp-permissions-anonymous-user-uploads.html

http://bashcurescancer.com/installing-ftp-vsftpd-on-rhel-50-centos.html

SSIM AD Integration

SSIM can integrate with multiple AD domains (do not have to be in same forest). The documentation in the Admin guide and the official KB is sparse on how to do that though currently.

Suffice it to say I was able to connect to a Windows 2003 Enterprise SP2 (not R2), as well as a Win 2003 R2 domains, in separate forests.

The trick is to make sure those domain controllers have been assigned a server certificate (not the root cert). Verify that certificate has "Server Authentication" in the Intended Purposes column of the Certificates snapin, where it should be under Personal/Certificates. When you open the certificate in the General tab it should read "You have a private key that corresponds to this certificate."

Troubleshooting steps: test connectivity using LDP.exe from the support tools installation. Also use the following command to test from the SSIM itself: openssl s_client -connect [your DC FQDN]:636

The following guide was perfect. Make sure to note the part about exporting the cert to Base-64 encoded binary X.509 (.CER). http://www.tools4ever.com/documentation/user-management-resource-administrator/ldap_ad_secure.htm?content=7030

The Symantec KB: http://www.symantec.com/business/support/index?page=content&id=TECH123285

iptables port manipulation

You may find there's no /etc/sysconfig/iptables in a CentOS install with iptables disabled upon install, but you want to redirect a port, e.g. to get port 80 working in tomcat.

Do:

service iptables start
iptables -t nat -A PREROUTING -p tcp --dport 80 -j REDIRECT --to-port 8080
iptables-save > /etc/sysconfig/iptables
service iptables restart

To open port 80, add the following to the *filter section before COMMIT:

-A INPUT -p tcp -m tcp --dport 80 -j ACCEPT

then service iptables restart

Numeric IP - SSIM Representation

IPs in the SSIM as of 4.7.3 are represented in both positive and negative form. To convert from an IP to this form, use the following, replacing [ipvariable] with your dotted quad IP:

SELECT CASE WHEN (IPNumber - 2147483648) > 0 THEN IPNumber -(2*2147483648) ELSE IPNumber END AS IPNUMBER2 FROM ( SELECT 16777216 *
CAST(LEFT(IPNumber1, LOCATE('.', IPNumber1, 1)-1) AS BIGINT) + 65536 * CAST(SUBSTR(IPNumber1, LOCATE('.', IPNumber1, 1) + 1, LOCATE('.',
IPNumber1,LOCATE('.', IPNumber1, 1) +1) - LOCATE('.', IPNumber1, 1) - 1) AS BIGINT) + 256 * CAST(SUBSTR(IPNumber1, LOCATE('.',
IPNumber1,LOCATE('.', IPNumber1, 1) +1) + 1, LOCATE('.', IPNumber1, LOCATE('.', IPNumber1,LOCATE('.', IPNumber1, 1) +1) +1) - LOCATE('.',
IPNumber1,LOCATE('.', IPNumber1, 1) +1) - 1) AS BIGINT) + CAST(RIGHT(IPNumber1, LENGTH(IPNumber1) - LOCATE('.', IPNumber1,
LOCATE('.', IPNumber1,LOCATE('.', IPNumber1, 1) +1) +1)) AS BIGINT) AS IPNumber FROM (SELECT '[ipvariable]' AS IPNumber1 FROM
SYSIBM.SYSDUMMY1) AS LOCXTMP0) AS TEMP2

ref: http://stackoverflow.com/questions/6695428/how-to-split-an-ip-address-string-in-db2-sql-for-converting-to-ip-number

SSIM Red Queues

When you see all three left red bars completely red, and the EPS counts are 0, here are some troubleshooting tips:

- Check the Sesa Agent if it's a collector server
- SSH into the collector server in question
- "su -" (change to root)
- tail the collector log which produces the most EPS or highest spikes on this collector
e.g. "tail -200 /opt/Symantec/sesa/Agent/logs/stonegate.log"
- If you see WARN entries containing "the agent queue is full" you may need to adjust the agent queue configurations: SSIM console: System > Product Configurations > SES > STATE > SSIM Agent and Manager > Agent Configurations > Standard (or the configuration applied to this server) > Logging
- adjust the following properties as needed, e.g. to increase the queue size, flush size, decrease the queue flush time (so that events are cleared more frequently).
- Save the configuration (the agent should pick up the settings automatically)

- Check the Recv-Q and Send-Q of Netstat
- on both the archiver and the correlator do "netstat -anp | grep 100"
- look for signs of downstream bottlenecks
- Examples
- if archiver is queued up sending to correlator:10010, check the correlator queues. E.g. if the correlator is queued up sending to 127.0.0.1:10080, there may be a problem with the asset service. Check assetsvc.log for errors. Try restarting the assetsvc service, and monitor the netstat -anp | grep 100 for movement in the queues.
As db2admin, cd ~/sqllib/bin
db2 connect to SESA
db2 list applications show detail
Check the SSIM-ASSET application for signs that it is currently Executing. Let it run for a while to see if it finishes.
- on correlator, check "netstat -anp | grep 555" for large number of FIN_WAIT2
- on the correlator check the icesvc.log, assetsvc.log, and rulesvc.log for recent errors (see /opt/Symantec/simserver/logs)
- Disable any rules temporarily that contain references to lookup tables to determine whether any of these rules are causing the backlog.
- Check swap space usage on Correlator. If there is any Swap used, reboot the server.
- Check SSIM Event logs for warnings or errors; query all recent logs where product = Symantec Security Information Manager or product = SSIM System
- Do a query on Event Type ID = Conclusion Updated. Verify that not more than a few events a minute are being created
- Check top on both archiver and correlator. Look for processes whose VIRT is close to the same size as RES. This process may be overworked. The memory allocation can be increased in the /opt/Symantec/simserver/svclauncher.cfg config.
- Investigate whether simserver (correlator service) is overloaded.
- ascertain the pid of the simserver in top
- do lsof | grep | awk '{print $7,$1,$2,$3,$4,$5,$6,$8,$9}' | sort -n
- look for any unusually large files in output. e.g. .que files larger than 10MB might indicate an overworked correlator.
- Do an advanced SQL query in the UI to determine whether there are any unusually large number of incidents for particular rule: SELECT count(incident_code),incident_code FROM SYMCMGMT.SYMC_IMR_INCIDENT_LIST_VIEW group by incident_code
order by count(incident_code) desc
or: SELECT desc_id, sum(event_count) as sumevtcnt FROM symc_sim_conclusion
where modified_time >= (current timestamp - 1 DAY) group by desc_id
order by sumevtcnt desc
- Disable any rules with more than 50,000 events in one day. For example, a system rule that can cause issues if not adapted is Internal Port Sweep. Simserver process memory will be exhausted typically after 24 hours, and Queues will increase steadily in /opt/Symantec/simserver/queues/ice/input. IIRC the max is three 64MB files here. If there are more than one file, the icesvc is backed up due to the amount of event tracking updates waiting to be pushed to the database. Verify that SSIM-ICE is "UOW Executing" in db2 list applications show detail.
- Also check Events > System Queries > SSIM > SSIM System > Count of Conclusions by Rule Name
- Look for FIN_WAITs associated with large queues in a frozen state in netstat for the simserver process (get pid from `status`). E.g.:
tcp 80384 0 ::ffff:127.0.0.1:10010 ::ffff:127.0.0.1:36446 ESTABLISHED 6131/java
tcp 0 145961 ::ffff:127.0.0.1:36446 ::ffff:127.0.0.1:10010 FIN_WAIT1 -
This could indicate a memory resource problem in the simserver service or another service. Check the target port, and cross reference the admin guide with the service in question. e.g. 10010 is simserver, 55562 is icesvc, etc.
If it's really in frozen state, the process may have crashed (i.e. you see 0 EPS). If the Recv-Q/Send-Q varies and EPS varies, then this could be indicative of High IO Waits. Check the IOWait in the Statistics tile of the GUI. Also check vmstat 1. Sustained IOWaits of more than 2% could indicate the drive is not fast enough.

Install syslog-ng 3.3 on CentOS 5.5

Download from Balabit: syslog-ng_3.3.0bet1.tar.gz or syslog-ng_3.3.0beta2.tar.gz (see http://www.balabit.com/downloads/files?path=/syslog-ng/sources/3.3.0beta2/source) and unpack.
to get the latest, install git:
first update the repo to obtain epels:
rpm -Uvh http://download.fedora.redhat.com/pub/epel/5/i386/epel-release-5-3.noarch.rpm
then yum install git
or download the snapshot from http://git.balabit.hu/?p=bazsi/syslog-ng-3.3.git;a=tree
and untar over the above.

There may be prereqs on your system.

Download these rpms and install. :

rpm -Uvh kernel-headers-2.6.18-238.el5.i386.rpm
rpm -Uvh glibc-headers-2.5-58.i386.rpm
rpm -Uvh glibc-devel-2.5-58.i386.rpm
rpm -Uvh gcc-4.1.2-50.el5.i386.rpm
rpm -Uvh zlib-1.2.3-4.el5.i386.rpm
rpm -ivh zlib-devel-1.2.3-4.el5.i386.rpm
rpm -Uvh libffi-3.0.5-1.el5.kb.i386.rpm
rpm -Uvh libffi-devel-3.0.5-1.el5.kb.i386.rpm

Then install glib
wget ftp://ftp.gtk.org/pub/glib/2.29/glib-2.29.4.tar.bz2
tar jxvf glib-2.29.4.tar.bz2
cd glib-2.29.4
./configure
make
make install
done with installing glib prerequisite

Before install syslog-ng 3.3 beta, do
export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig:$PKG_CONFIG_PATH
so that eventlog is found.

also do yum install openssl-devel
or

rpm -Uvh e2fsprogs-devel-1.39-23.el5_5.1.i386.rpm
rpm -Uvh keyutils-libs-devel-1.2-1.el5.i386.rpm
rpm -Uvh libselinux-devel-1.33.4-5.7.el5.i386.rpm
rpm -Uvh libsepol-devel-1.15.2-3.el5.i386.rpm
rpm -Uvh libselinux-devel-1.33.4-5.7.el5.i386.rpm
rpm -Uvh krb5-devel-1.6.1-55.el5.i386.rpm
rpm -Uvh openssl-devel-0.9.8e-12.el5_5.7.i386.rpm

also:
cd eventlog-0.2.12/
./configure
make
make install
then enter syslog-ng dir and
./configure --enable-pcre --disable-mongodb --disable-ipv6
make
make install
Note: To install with PCRE enabled, must first yum install pcre-devel
After that, some more setup:
mkdir /usr/local/var
cd /root/syslog-ng/syslog-ng-3.3.0beta1/contrib/
cp rhel-packaging/syslog-ng.init /etc/init.d/syslog-ng
cp rhel-packaging/syslog-ng.conf /usr/local/etc/
cp rhel-packaging/syslog-ng.logrotate /etc/logrotate.d/syslog-ng
chmod 755 /etc/init.d/syslog-ng
vi /etc/init.d/syslog-ng
Then change the binary to "/usr/local/sbin/syslog-ng"
Then start the service:
chkconfig syslog off or chkconfig rsyslog off as applicable

chkconfig syslog-ng on

service syslog stop

service syslog-ng start

Then edit /usr/local/etc/syslog-ng.conf
add options { threaded(yes); }; to the top, and any other configs.

service syslog-ng restart

CentOS 5.5 Kernel Upgrade and iotop Install

To run iotop.py you need a kernel greater than what CentOS provides. The following steps guide this upgrade, so that you can use iotop.

cd /usr/src
wget http://www.kernel.org/pub/linux/kernel/v2.6/linux-2.6.39.1.tar.bz2
tar xjf linux-2.6.39.1.tar.bz2
ln -s linux-2.6.39.1 linux
cd linux
make mrproper
cp /boot/config-`uname -r` .config
yes "" | make oldconfig (this updates .config to all defaults)

edit .config, adding or updating the following options:
CONFIG_SYSFS_DEPRECATED=y
CONFIG_SYSFS_DEPRECATED_V2=y

make
make modules_install
make install
reboot

rpm -Uvh http://download.fedora.redhat.com/pub/epel/5/i386/python-ctypes-1.0.2-2.el5.i386.rpm
wget http://guichaz.free.fr/iotop/files/iotop-0.4.3.tar.gz
tar zxvf iotop-0.4.3.tar.gz
cd iotop-0.4.3
./iotop.py


Disclaimer: Perform the above instructions at your own risk.

For instructions on how to make the rpm (so the lengthy compilation can be done once for future installs), see http://www.howtoforge.com/kernel_compilation_centos
Note: will need to install rpm-build and ncurses-devel

Refs:

http://www.centos.org/modules/newbb/viewtopic.php?topic_id=8488

http://forum.openvz.org/index.php?t=msg&goto=41203&#msg_41203

http://guichaz.free.fr/iotop/