Style "Newsletter, no line" and the DVWP

Tried converting (via "convert to XSLT Data View" in Sharepoint Designer 2007) a ListViewWebPart whose style was set to "NewsLetter, no line" and it would not work. The resulting XSLT produced mangled HTML tables. So trying my luck I tried converting the plain Newsletter style using the same approach. This had better results, although, there was an odd table cell on the left, which appeared due the condition ddwrt:GetVar('NumColumns')='1' being true. This is odd, because NumColumns in my list view should be four not one. Further searching the code showed that NumColumns is set as the string length of "Columns," i.e. ddwrt:SetVar('NumColumns', string(string-length(ddwrt:GetVar('Columns')))) and Columns is set to a lone period (dot).
After analyzing the CAML of the LVWP, which should be the basis for the XSLT, I found that in the CAML version, the assignment of Columns is set in a Fields element, which according to the View Schema on MSDN contains a loop over the view fields. This explains the difference: At the end of the loop, Columns is set to "....".
So what I did to patch the XSLT was simply replace the single dot with four dots in the assignment of Columns, which took care of the odd cell, and probably some other anomalies.
There is still some more fixing up to do, e.g.:
- delete the header cell for the field that displays on its own line
- setting column headers to be sortable.
- comment out the ms-alternating class assignment because it's applied unevenly to tr tags, unless you want to convert to "No line" (instructions below)
- explicitely set the class for content cells. Designer sets them incorrectly, e.g. TD Class="{$IDACAVMC}, which is a variable that is based on a List View CAML property, which no longer applies, apparently. So, icon cell would be ms-vb-icon; title cell would be set to ms-vb-title, user would be ms-vb-user; and the text cell on its own row has already been hard coded for us by the Designer ListViewWebPart to DataFormWebPart conversion process.

Now, to transform this into a "Newsletter, no line" style, do the following:
- comment out the TR tag containing the TD class="ms-nlline"
- replace the html encoded starting TR tags (all <xsl:text disable-output-escaping="yes">&lt;tr&gt;</xsl:text>) with actual TR tags; note: must add the closing TR tag, because it was not added during conversion. To find out where to insert the closing TR for the main row, select the row in SPD, backtrack to the last <xsl:choose> that is directly above the two repeated field name comments for the field that will appear on its own line. Place the closing TR above the two field name comments.
- add the conditional ms-alternating attribute to those TR tags.
Note: There should be two xsl:text replacements (the second one being in the vicinity of "ms-nlline" below the @Remarks=) and three conditional (if statements)
ms-alternating attributes added based on a four column table with one field on its own line: two are added to the replaced TRs, and one added to the TR above the removed ms-nlline row.

Copying Discussion Items to Document Libary

The Requirement is to copy discussion items to another document library.

Code ideas borrowed from: http://stackoverflow.com/questions/468469/how-do-you-upload-a-file-to-a-document-library-in-sharepoint, http://sharepoint.stackexchange.com/questions/20216/iterate-through-discussion-list, http://www.sharepoint-tips.com/2011/11/event-handler-to-archive-items-when.html

Sample code containing the general idea:

using System;
using System.Runtime.InteropServices;
using System.Web.UI;
using System.Web.UI.WebControls;
using System.Web.UI.WebControls.WebParts;
using System.Xml.Serialization;

using Microsoft.SharePoint;
using Microsoft.SharePoint.WebControls;
using Microsoft.SharePoint.WebPartPages;

namespace WebPart3
{
[Guid("26d38752-d4ce-456e-88f6-496d213627d4")]
public class WebPart3 : System.Web.UI.WebControls.WebParts.WebPart
{
string msg1;
Button saveTitle;
TextBox newTitle;
Label label;

public WebPart3()
{
SPSite site1 = SPContext.Current.Site;

msg1 = "(1.03)" ;

}

public void saveTitle_click(object sender, EventArgs e)
{
//this.Title = newTitle.Text;
string tempprint = "";
SPWeb myweb = SPContext.Current.Web;
SPListCollection lists = myweb.Lists;
SPList mylist;
SPListItem sli;
string labeltext = "";
SPList targetList;

try
{

// SPList targetList = lists["Discussion Archive"];
// SPListItem newItem = targetList.Items.Add();

mylist = lists[newTitle.Text];
targetList = lists["Discussion Archive"];
// targetList = lists["DiscussionDocs"];

// SPQuery query = new SPQuery();
//query.RowLimit = 10;
// SPListItemCollection posts = mylist.GetItems(query);

//this. .SaveProperties = true;
try
{
labeltext += " entering foreach";
foreach (SPListItem item in mylist.Folders)
{
SPListItem newItem = targetList.Items.Add();
//copy the list item to the target
foreach (SPField f in item.Fields)
{
if (!f.ReadOnlyField && newItem.Fields.ContainsField(f.InternalName))
newItem[newItem.Fields.GetFieldByInternalName(f.InternalName).Id] = item[f.Id];
}
//copy "special" read only fields that can be written to
newItem["Created By"] = item["Created By"];
newItem["Modified By"] = item["Modified By"];
newItem["Modified"] = item["Modified"];
newItem["Created"] = item["Created"];
newItem.SystemUpdate(false);
CopyAttachments(item, newItem);


labeltext += item["Body"] + ",";
sli = item;
tempprint = (string)sli["Created By"];
}
labeltext += " existing foreach";
try
{
this.Title = "yo" + mylist.ItemCount + " by: " + tempprint;

}
catch (Exception ex)
{
this.Title = "Error3: " + ex.Message;
label.Text = ex.StackTrace;

}
}
catch (Exception ex)
{

this.Title = "Error2: " + ex.Message;
label.Text = ex.StackTrace;
}
}
catch (Exception ex)
{
this.Title = "Error: " + ex.Message;
label.Text = ex.StackTrace;
}
label.Text += labeltext + "the end";
// this.SetPersonalizationDirty();
}

private void CopyAttachments(SPListItem sourceItem, SPListItem targetItem)
{
SPWeb web = SPContext.Current.Web;

SPFolder myLibrary = web.Folders["DiscussionDocs"];
Boolean replaceExistingFiles = true;

try
{
//get the folder with the attachments for the source item
SPFolder sourceItemAttachmentsFolder = sourceItem.Web.Folders["Lists"].SubFolders[sourceItem.ParentList.Title]
.SubFolders["Attachments"].SubFolders[sourceItem.ID.ToString()];
foreach (SPFile file in sourceItemAttachmentsFolder.Files)
{
byte[] binFile = file.OpenBinary();
targetItem.Attachments.AddNow(file.Name, binFile);
this.label.Text += file.Name + " attached to target ";
SPFile spfile = myLibrary.Files.Add(file.Name, binFile, replaceExistingFiles);
SPListItem newfileitem = spfile.Item;
newfileitem["Created"] = sourceItem["Created"];
newfileitem["Modified By"] = sourceItem["Modified By"];
newfileitem["Modified"] = sourceItem["Modified"];
newfileitem["Created By"] = sourceItem["Created By"];
newfileitem["Title"] = sourceItem["Title"];
//newfileitem.SystemUpdate(false);
newfileitem.Update();

myLibrary.Update();
}
}
catch (Exception ex)
{
this.Title = "Error: " + ex.Message;
}
}

protected override void CreateChildControls()
{
base.CreateChildControls();

// TODO: add custom rendering code here.
label = new Label();
label.Text = "Hello World" + msg1;
this.Controls.Add(label);

//Create text box
newTitle = new TextBox();
newTitle.Text = "myxmdiscuss";
this.Controls.Add(newTitle);

//Create Button
saveTitle = new Button();
saveTitle.Text = "Set Web Partt Title";
saveTitle.Click += new EventHandler(this.saveTitle_click);
Controls.Add(saveTitle);
}
}
}

And here's the Powershell:

$error.clear()

[System.reflection.Assembly]::LoadWithPartialName("Microsoft.SharePoint")

$site = new-object Microsoft.SharePoint.SPSite("https://site")
$web = $site.OpenWeb()

$lists = $web.Lists;

$lista = $lists['Team Discussion']
$listb = $web.Folders['MyDocLib']

foreach ($item in $lista.Folders)
{
Try
{
$sourceItemAttachmentsFolder = $item.Web.Folders["Lists"].SubFolders[$item.ParentList.Title].SubFolders["Attachments"].SubFolders[$item.ID.ToString()];
foreach ($file in $sourceItemAttachmentsFolder.Files)
{
$binfile = $file.OpenBinary()
"Attached to target " + $item["Title"]
$spfile = $listb.Files.Add($file.Name, $binfile, 1)
$newfileitem = $spfile.Item
foreach ($f in $item.Fields)
{
if (!$f.ReadOnlyField -and $newfileitem.Fields.ContainsField($f.InternalName))
{
$newfileitem[$newfileitem.Fields.GetFieldByInternalName($f.InternalName).Id] = $item[$f.Id];
}

$newfileitem["Created By"] = $item["Created By"];

$newfileitem["Created"] = $item["Created"];

$newfileitem["Modified By"] = $item["Modified By"];

$newfileitem["Modified"] = $item["Modified"];

$newfileitem.Update();

}
}

}
Catch
{

"caught a system exception"
}

$error

}

Note: Minimum permissions to execute the ps1 is local server administrator + Site owner.

~SiteCollection in ParameterBinding

Here's another workaround. You would like to assign the ~SiteCollection token value (or any token) to a ParameterBinding so that your XSLT can see that value and use it say in image paths, etc.

After finding a tip <a href="http://stackoverflow.com/questions/609943/dynamically-set-the-defaultvalue-of-a-parameterbinding-in-a-dataformwebpart">here</a>, the solution may be as follows.

- Put a hidden html input control with a runat=server attribute in your content place holder, e.g. <input id="siteColVal">" type=hidden name=siteColVal runat="server">
(Note, an asp:TextBox control did not work for me).
- Add a parameterbinding like: <parameterbinding location="Control(siteColVal, value)" name="SiteColRoot" defaultvalue="">
- then of course a <?xml:namespace prefix = xsl /><xsl:param name="SiteColRoot">theooroot</xsl:param> in your XSLT

Worked for me!

Sort DataFormWebPart by "Order" Field

The only way I know to do this SPD 2007 is through some manual steps. BTW, the "order" field is determined by the action taken by user, which is made available via the "Allow users to order items in this view?" option for the view.

There are actually a couple of ways:

- Drag your list from the Web Part List from the Web Parts Task pane onto a web part zone.
- Then right click on web part and "Convert to XSLT Data View" which will convert the ListViewWebPart to DataFormWebPart with all the desired attributes (such as sorting) preset.

Alternately, if you created the web part as a Custom List Form from the start (i.e. with XSLT embedded), then you can edit the CAML manually.

- First open Common Data View Tasks and set it to sort by a common field such as ID.
- Then edit the SPDataSource SelectCommand, by changing "<OrderBy><FieldRef Name="ID"" to "<OrderBy><FieldRef Name="Order""

Deploy List Definition

- Start SharePoint Solution Generator 2008 (comes with Visual Studio 2008 Extensions)

- Select List Definition

- click Next; if you get an error "VSeWSS Service Error: No SharePoint Site exists at the specified URL", you can ignore and click OK

- Then enter the URL, next, select the list and finish

- Open the resultant project in Visual Studio 2008 and Build - Package solution.

- Run the generated setup.bat from command line

- You may get an error if the current server does not have the Windows SharePoint Services Web application started, if so start it, and try again.

- Ignore the errors (which you might get if this is a farm with more than one web front end) concerning feature activation as we will deploy and activate next

- Open Solution management in Central Administration and Deploy the new solution (wsp file)

- another option is from the command line using the allcontenturls flag. Note the setup.bat sets the -local flag, which only applies to single server farms. However, you can use the -local flag to deploy to only the local WFE, if global deployment isn't working, e.g. because some WFEs are down.

If you intend to use the -local flag, you will need to edit setup.bat and add the following to the LDeploy section after the deploysolution operation

echo installing feature FeatureName ...

"%SPAdminTool%" -o installfeature -name "FeatureName"

- Now you can activate the lists included with the solution individually from the command line. You can verify that the solution was deployed by checking under C:\Program Files\Common Files\Microsoft Shared\web server extensions\12\TEMPLATE\FEATURES\ for the names of your lists

- To activate the list/feature on a given site, do: "C:\Program Files\Common Files\Microsoft Shared\web server extensions\12\TEMPLATE\FEATURES\bin\stsadm.exe" -o activatefeature -id [get the id from the setup.bat file] -url http://urlofyoursite:port

That's it! Now the list is ready to be selected during list creation.

As far as activating the list/feature on all sites, there are some powershell solutions out there, e.g. http://bable.cybermarshall.com/2009/01/17/using-powershell-to-activate-a-feature-across-all-sharepoint-2007wss-30-sites-and-subsites/

(haven't tried it yet)

ref: http://blogit.create.pt/blogs/andrevala/archive/2008/02/17/SharePoint-2007-Deployment_3A00_-Creating-Solutions.aspx

Group Email in MOSS

There are a couple of gotchas in Sharepoint 2007 you have to watch for when configuring group emails.

First, if you are trying to configure alerts to an AD security group email, make sure that the security group appears under the permissions for the list, either individually or in a group. Then you can create the alert by replacing your email with that of the security group. Although, at this time, I wouldn't know where to find this alert, if I wanted to delete it.

Secondly, there a few things to watch for when trying to send an email to a Sharepoint group. First you have to email enable the sites; do it in CA under Operations, Incoming Email Settings. Then in your group's settings, create the e-mail distribution list. After submitting you will notice below "Distribution list e-mail address:" a message "Distribution list status: Creation request pending." This means you must approve the request in CA, Operations, Approve/reject Distribution Groups. When that's done, you can email the group. However, what I had to do was verify the actual email address used as shown in Active Directory Users and Computers under the OU where Sharepoint created the distribution group. In my case, the email as shown in AD was based on the domain, i.e. mygroup@mydomain.com. Whereas in Sharepoint in the group settings, was shown mygroup@server.mydomain.com. Make sure the Windows SharePoint Services Timer service is running or else the email you send to mygroup@mydomain.com (e.g. emailing it from Outlook) may never be sent.

There is still a problem with regard to adding this group email to an email action in a Sharepoint Designer built workflow. The email does not send. The NDA email contains "You do not have permission to send to this recipient." I have tried removing connection and relay restrictions in the SMTP virtual server in Exchange System Manager. I also tried configuring Sharepoint to send via a local SMTP server that relays to Exchange. But I get the same NDA error. I have had no luck getting group email to work in the three-state workflow as well, so maybe it's related to group functionality. If I select the group from the Select Users picker in Define E-mail Message SPD workflow, I do not get an NDA, but no email is sent. If I type in the group's email address, as it appears in AD, then I get the NDA. If I use the long form email address, no email is sent, nor an NDA.

MOSS - Fix "subsite1 is already in use" Error

I created a Site collection, selecting the default Team Site Collaboration template. I then logged into the new site and attempted to create a subsite "subsite1" for which I selected the "News Site" Publishing template. This produced the following error: "sharepoint server publishing infrastructure feature must be activated". So then I activate this feature in the Site Collection Features page. I then go back and create subsite1. This gave me the error "The Web site address "/subsite1" is already in use. Funny that apparently the site was created despite the non-activated feature (bug -- probably fixed by now).

To fix, just go to the site collection root site, then Site Settings, Site hierarchy, click Manage next to subsite1 (e.g.), then click Delete this site.

Now I can create that site called subsite1. Moss 2007.

Multi-threading SSIM Event Search

To ensure your search is multithreaded (and thus faster) as of 4.7.4.x you must break your search into separate archives. Note, that SSIM is agnostic when it comes to archives: an "archive" is simply a directory with .edx, .key, .ndx, .sar, .tdx, .vdx files. However, an official archive will have an indexed_event_fields.txt file in it.

One quick way to do this is unzip the Archive_CLI tool and run the search from the command line, inputing archives separated by a comma, e.g.:date;java -server -Xmx512m -verbose:gc -XX:+UseConcMarkSweepGC -XX:+PrintGCTimeStamps -XX:+PrintGCDetails -jar simsar.jar -a /eventarchive/pixarchive1/,/eventarchive/pixarchive2/,/eventarchive/default,/eventarchive/ssimlogs -q "destination_ip = \"192.168.1.1\" & (destination_port = 80 | destination_port = 8080)" -c -S "," -r events.csv -V;date
Note the extra flags are for monitoring garbage collection. Check the threads in top to verify parallelism.

Thread Dump of SSIM Service

To get a global thread dump of a SSIM service, find the log_file parameter for the service in svclauncher.cfg to determine the log file that will contain the thread dump. Then issue a status to get the PID of the service in question, e.g. the rulesvc. Then while the SSIM is performing some interesting activity, type kill -SIGQUIT <PID>. Finally, open the log file to examine the thread dump and stack traces.

Quick FTP Setup

This is a quick and dirty setup guide to enabling ftp uploads for your server to take advantage of FTP file transfer speeds which are much faster than SFTP/SCP. It only applies to servers in a secured lab environment, and should never be performed on production servers or public-facing servers.

- check whether vsftpd is installed:

rpm -q vsftpd

yum install vsftpd

- configure it:

chmod 777 /var/ftp/pub

vi /etc/vsftpd/vsftpd.conf

change anon_upload_enable and anon_mkdir_write_enable to YES; add line with: ftp_username=nobody (so that will prompt for user in RedHat linux)

service vsftpd start

ftp login using user anonymous with blank password



Note:

you may need to disable iptables and selinux.

refs:

http://nixcraft.com/getting-started-tutorials/725-secure-vsftpd-ftp-permissions-anonymous-user-uploads.html

http://bashcurescancer.com/installing-ftp-vsftpd-on-rhel-50-centos.html

SSIM AD Integration

SSIM can integrate with multiple AD domains (do not have to be in same forest). The documentation in the Admin guide and the official KB is sparse on how to do that though currently.

Suffice it to say I was able to connect to a Windows 2003 Enterprise SP2 (not R2), as well as a Win 2003 R2 domains, in separate forests.

The trick is to make sure those domain controllers have been assigned a server certificate (not the root cert). Verify that certificate has "Server Authentication" in the Intended Purposes column of the Certificates snapin, where it should be under Personal/Certificates. When you open the certificate in the General tab it should read "You have a private key that corresponds to this certificate."

Troubleshooting steps: test connectivity using LDP.exe from the support tools installation. Also use the following command to test from the SSIM itself: openssl s_client -connect [your DC FQDN]:636

The following guide was perfect. Make sure to note the part about exporting the cert to Base-64 encoded binary X.509 (.CER). http://www.tools4ever.com/documentation/user-management-resource-administrator/ldap_ad_secure.htm?content=7030

The Symantec KB: http://www.symantec.com/business/support/index?page=content&id=TECH123285

iptables port manipulation

You may find there's no /etc/sysconfig/iptables in a CentOS install with iptables disabled upon install, but you want to redirect a port, e.g. to get port 80 working in tomcat.

Do:

service iptables start
iptables -t nat -A PREROUTING -p tcp --dport 80 -j REDIRECT --to-port 8080
iptables-save > /etc/sysconfig/iptables
service iptables restart

To open port 80, add the following to the *filter section before COMMIT:

-A INPUT -p tcp -m tcp --dport 80 -j ACCEPT

then service iptables restart

Numeric IP - SSIM Representation

IPs in the SSIM as of 4.7.3 are represented in both positive and negative form. To convert from an IP to this form, use the following, replacing [ipvariable] with your dotted quad IP:

SELECT CASE WHEN (IPNumber - 2147483648) > 0 THEN IPNumber -(2*2147483648) ELSE IPNumber END AS IPNUMBER2 FROM ( SELECT 16777216 *
CAST(LEFT(IPNumber1, LOCATE('.', IPNumber1, 1)-1) AS BIGINT) + 65536 * CAST(SUBSTR(IPNumber1, LOCATE('.', IPNumber1, 1) + 1, LOCATE('.',
IPNumber1,LOCATE('.', IPNumber1, 1) +1) - LOCATE('.', IPNumber1, 1) - 1) AS BIGINT) + 256 * CAST(SUBSTR(IPNumber1, LOCATE('.',
IPNumber1,LOCATE('.', IPNumber1, 1) +1) + 1, LOCATE('.', IPNumber1, LOCATE('.', IPNumber1,LOCATE('.', IPNumber1, 1) +1) +1) - LOCATE('.',
IPNumber1,LOCATE('.', IPNumber1, 1) +1) - 1) AS BIGINT) + CAST(RIGHT(IPNumber1, LENGTH(IPNumber1) - LOCATE('.', IPNumber1,
LOCATE('.', IPNumber1,LOCATE('.', IPNumber1, 1) +1) +1)) AS BIGINT) AS IPNumber FROM (SELECT '[ipvariable]' AS IPNumber1 FROM
SYSIBM.SYSDUMMY1) AS LOCXTMP0) AS TEMP2

ref: http://stackoverflow.com/questions/6695428/how-to-split-an-ip-address-string-in-db2-sql-for-converting-to-ip-number

SSIM Red Queues

When you see all three left red bars completely red, and the EPS counts are 0, here are some troubleshooting tips:

- Check the Sesa Agent if it's a collector server
- SSH into the collector server in question
- "su -" (change to root)
- tail the collector log which produces the most EPS or highest spikes on this collector
e.g. "tail -200 /opt/Symantec/sesa/Agent/logs/stonegate.log"
- If you see WARN entries containing "the agent queue is full" you may need to adjust the agent queue configurations: SSIM console: System > Product Configurations > SES > STATE > SSIM Agent and Manager > Agent Configurations > Standard (or the configuration applied to this server) > Logging
- adjust the following properties as needed, e.g. to increase the queue size, flush size, decrease the queue flush time (so that events are cleared more frequently).
- Save the configuration (the agent should pick up the settings automatically)

- Check the Recv-Q and Send-Q of Netstat
- on both the archiver and the correlator do "netstat -anp | grep 100"
- look for signs of downstream bottlenecks
- Examples
- if archiver is queued up sending to correlator:10010, check the correlator queues. E.g. if the correlator is queued up sending to 127.0.0.1:10080, there may be a problem with the asset service. Check assetsvc.log for errors. Try restarting the assetsvc service, and monitor the netstat -anp | grep 100 for movement in the queues.
As db2admin, cd ~/sqllib/bin
db2 connect to SESA
db2 list applications show detail
Check the SSIM-ASSET application for signs that it is currently Executing. Let it run for a while to see if it finishes.
- on correlator, check "netstat -anp | grep 555" for large number of FIN_WAIT2
- on the correlator check the icesvc.log, assetsvc.log, and rulesvc.log for recent errors (see /opt/Symantec/simserver/logs)
- Disable any rules temporarily that contain references to lookup tables to determine whether any of these rules are causing the backlog.
- Check swap space usage on Correlator. If there is any Swap used, reboot the server.
- Check SSIM Event logs for warnings or errors; query all recent logs where product = Symantec Security Information Manager or product = SSIM System
- Do a query on Event Type ID = Conclusion Updated. Verify that not more than a few events a minute are being created
- Check top on both archiver and correlator. Look for processes whose VIRT is close to the same size as RES. This process may be overworked. The memory allocation can be increased in the /opt/Symantec/simserver/svclauncher.cfg config.
- Investigate whether simserver (correlator service) is overloaded.
- ascertain the pid of the simserver in top
- do lsof | grep | awk '{print $7,$1,$2,$3,$4,$5,$6,$8,$9}' | sort -n
- look for any unusually large files in output. e.g. .que files larger than 10MB might indicate an overworked correlator.
- Do an advanced SQL query in the UI to determine whether there are any unusually large number of incidents for particular rule: SELECT count(incident_code),incident_code FROM SYMCMGMT.SYMC_IMR_INCIDENT_LIST_VIEW group by incident_code
order by count(incident_code) desc
or: SELECT desc_id, sum(event_count) as sumevtcnt FROM symc_sim_conclusion
where modified_time >= (current timestamp - 1 DAY) group by desc_id
order by sumevtcnt desc
- Disable any rules with more than 50,000 events in one day. For example, a system rule that can cause issues if not adapted is Internal Port Sweep. Simserver process memory will be exhausted typically after 24 hours, and Queues will increase steadily in /opt/Symantec/simserver/queues/ice/input. IIRC the max is three 64MB files here. If there are more than one file, the icesvc is backed up due to the amount of event tracking updates waiting to be pushed to the database. Verify that SSIM-ICE is "UOW Executing" in db2 list applications show detail.
- Also check Events > System Queries > SSIM > SSIM System > Count of Conclusions by Rule Name
- Look for FIN_WAITs associated with large queues in a frozen state in netstat for the simserver process (get pid from `status`). E.g.:
tcp 80384 0 ::ffff:127.0.0.1:10010 ::ffff:127.0.0.1:36446 ESTABLISHED 6131/java
tcp 0 145961 ::ffff:127.0.0.1:36446 ::ffff:127.0.0.1:10010 FIN_WAIT1 -
This could indicate a memory resource problem in the simserver service or another service. Check the target port, and cross reference the admin guide with the service in question. e.g. 10010 is simserver, 55562 is icesvc, etc.
If it's really in frozen state, the process may have crashed (i.e. you see 0 EPS). If the Recv-Q/Send-Q varies and EPS varies, then this could be indicative of High IO Waits. Check the IOWait in the Statistics tile of the GUI. Also check vmstat 1. Sustained IOWaits of more than 2% could indicate the drive is not fast enough.

Install syslog-ng 3.3 on CentOS 5.5

Download from Balabit: syslog-ng_3.3.0bet1.tar.gz or syslog-ng_3.3.0beta2.tar.gz (see http://www.balabit.com/downloads/files?path=/syslog-ng/sources/3.3.0beta2/source) and unpack.
to get the latest, install git:
first update the repo to obtain epels:
rpm -Uvh http://download.fedora.redhat.com/pub/epel/5/i386/epel-release-5-3.noarch.rpm
then yum install git
or download the snapshot from http://git.balabit.hu/?p=bazsi/syslog-ng-3.3.git;a=tree
and untar over the above.

There may be prereqs on your system.

Download these rpms and install. :

rpm -Uvh kernel-headers-2.6.18-238.el5.i386.rpm
rpm -Uvh glibc-headers-2.5-58.i386.rpm
rpm -Uvh glibc-devel-2.5-58.i386.rpm
rpm -Uvh gcc-4.1.2-50.el5.i386.rpm
rpm -Uvh zlib-1.2.3-4.el5.i386.rpm
rpm -ivh zlib-devel-1.2.3-4.el5.i386.rpm
rpm -Uvh libffi-3.0.5-1.el5.kb.i386.rpm
rpm -Uvh libffi-devel-3.0.5-1.el5.kb.i386.rpm

Then install glib
wget ftp://ftp.gtk.org/pub/glib/2.29/glib-2.29.4.tar.bz2
tar jxvf glib-2.29.4.tar.bz2
cd glib-2.29.4
./configure
make
make install
done with installing glib prerequisite

Before install syslog-ng 3.3 beta, do
export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig:$PKG_CONFIG_PATH
so that eventlog is found.

also do yum install openssl-devel
or

rpm -Uvh e2fsprogs-devel-1.39-23.el5_5.1.i386.rpm
rpm -Uvh keyutils-libs-devel-1.2-1.el5.i386.rpm
rpm -Uvh libselinux-devel-1.33.4-5.7.el5.i386.rpm
rpm -Uvh libsepol-devel-1.15.2-3.el5.i386.rpm
rpm -Uvh libselinux-devel-1.33.4-5.7.el5.i386.rpm
rpm -Uvh krb5-devel-1.6.1-55.el5.i386.rpm
rpm -Uvh openssl-devel-0.9.8e-12.el5_5.7.i386.rpm

also:
cd eventlog-0.2.12/
./configure
make
make install
then enter syslog-ng dir and
./configure --enable-pcre --disable-mongodb --disable-ipv6
make
make install
Note: To install with PCRE enabled, must first yum install pcre-devel
After that, some more setup:
mkdir /usr/local/var
cd /root/syslog-ng/syslog-ng-3.3.0beta1/contrib/
cp rhel-packaging/syslog-ng.init /etc/init.d/syslog-ng
cp rhel-packaging/syslog-ng.conf /usr/local/etc/
cp rhel-packaging/syslog-ng.logrotate /etc/logrotate.d/syslog-ng
chmod 755 /etc/init.d/syslog-ng
vi /etc/init.d/syslog-ng
Then change the binary to "/usr/local/sbin/syslog-ng"
Then start the service:
chkconfig syslog off or chkconfig rsyslog off as applicable

chkconfig syslog-ng on

service syslog stop

service syslog-ng start

Then edit /usr/local/etc/syslog-ng.conf
add options { threaded(yes); }; to the top, and any other configs.

service syslog-ng restart

CentOS 5.5 Kernel Upgrade and iotop Install

To run iotop.py you need a kernel greater than what CentOS provides. The following steps guide this upgrade, so that you can use iotop.

cd /usr/src
wget http://www.kernel.org/pub/linux/kernel/v2.6/linux-2.6.39.1.tar.bz2
tar xjf linux-2.6.39.1.tar.bz2
ln -s linux-2.6.39.1 linux
cd linux
make mrproper
cp /boot/config-`uname -r` .config
yes "" | make oldconfig (this updates .config to all defaults)

edit .config, adding or updating the following options:
CONFIG_SYSFS_DEPRECATED=y
CONFIG_SYSFS_DEPRECATED_V2=y

make
make modules_install
make install
reboot

rpm -Uvh http://download.fedora.redhat.com/pub/epel/5/i386/python-ctypes-1.0.2-2.el5.i386.rpm
wget http://guichaz.free.fr/iotop/files/iotop-0.4.3.tar.gz
tar zxvf iotop-0.4.3.tar.gz
cd iotop-0.4.3
./iotop.py


Disclaimer: Perform the above instructions at your own risk.

For instructions on how to make the rpm (so the lengthy compilation can be done once for future installs), see http://www.howtoforge.com/kernel_compilation_centos
Note: will need to install rpm-build and ncurses-devel

Refs:

http://www.centos.org/modules/newbb/viewtopic.php?topic_id=8488

http://forum.openvz.org/index.php?t=msg&goto=41203&#msg_41203

http://guichaz.free.fr/iotop/

Squid Proxy Quick Start Guide

First install dependency: yum install gcc-c++

Download squid, e.g. the file squid-3.X.STABLEXX.tar.gz
tar xvzf squid-3.X.STABLEXX.tar.gz
change dirs
./configure --enable-icap-client
make
make install

Comment out the allow from local network:
sed -rie '/http_access allow localnet/ s/^/#/' /usr/local/squid/etc/squid.conf
Start up squid in interactive mode:
/usr/local/squid/sbin/squid -N -d1

Add the squid IP and port (3128) to the browser proxy configuration.
Try surfing to a website; you will see:


ERROR

The requested URL could not be retrieved

--------------------------------------------------------------------------------

The following error was encountered while trying to retrieve the URL: http://www.awgtek.com/


Access Denied.

Access control configuration prevents your request from being allowed at this time. Please contact your service provider if you feel this is incorrect.


Do:
Ctl-C
sed -rie '/http_access allow localnet/ s/^#//' squid.conf

/usr/local/squid/sbin/squid -N -d1

Browse to a site.

To check the access
do: tail /usr/local/squid/var/logs/access.log

DLP "Forwarding agent unavailable" error

If this error is appearing in the /var/log/mail.log of your upstream MTA, when initially setting up a DLP Network Prevent for Email in Forwarding mode, try recycling the new smtp prevent server, before trying anything else.

e.g. errors like (host #.#.#.51[#.#.#.51]
refused to talk to me: 421 4.4.1 Fatal: Forwarding agent unavailable. Closing connection.

Recycling the DLP email prevent server got the following architecture to finally start working:

Outlook --> (port 25) Ubuntu Postfix MTA --> (port 10025) DLP SMTP Prevent --> (port 25) Ubuntu Postfix MTA --> (port 587, TLS) GMail

The upstream MTA's main.cf was configured with smtp_use_tls=no, smtp_sasl_auth_enable=no; and with relayhost=(DLP_IP):10025

The DLP server's advanced configuration had RequestProcessor.MTAResubmitPort=25 (changed from default 10026). Next Hop Configuration was set to Forward with Disable MX lookup, and downstream local MTA's IP set in Hostnames.

Setup Postfix Ubuntu Gmail SMTP Relay

Refer to http://ubuntu-tutorials.com/2008/11/11/relaying-postfix-smtp-via-smtpgmailcom/

On clean Ubuntu (natty) 11.04 (with ssh and aptitude installed, and as root)

aptitude install postfix libsasl2 ca-certificate libsasl2-modules

During Postfix Configuration Package configuration wizard
On first page, choose "Internet with smarthost"
For SMTP relay host enter: [smtp.gmail.com]:587
(brackets are to avoid mx lookups)

paste in /etc/postfix/main.cf:

smtp_sasl_auth_enable = yes
smtp_sasl_password_maps = hash:/etc/postfix/sasl_passwd
smtp_sasl_security_options = noanonymous
smtp_tls_CAfile = /etc/ssl/certs/Equifax_Secure_CA.pem
smtp_use_tls = yes

while still in main.cf, add your local IP network to the mynetworks variable

create /etc/postfix/sasl_passwd with the following contents:

[smtp.gmail.com]:587 user.name@gmail.com:password

do:

chmod 400 /etc/postfix/sasl_passwd
postmap /etc/postfix/sasl_passwd

/etc/init.d/postfix reload

Now test:

telnet postfixserverip 25
ehlo anything.com
mail from:me@whatever.com
rcpt to:someone@somewhere.com notify=success,failure
data
subject:Test from awgtek

This is a test message

.
quit

check someone@somewhere.com's email to see if message arrived
check for errors in /var/log/mail.log

After DLP DB install, start database if necessary

Was getting errors when verifying the Symantec Data Loss Prevention database. When I would type the command $ORACLE_HOME/bin/lsnrctl services LISTENER I would get The listener supports no services

This can happen if for some reason the database has not started.

Try starting the database:

$ sqlplus '/ as sysdba'
Then, enter
startup at the SQL prompt.

Once the database has started, verification succeeds!

See http://www.thegeekstuff.com/2009/01/oracle-database-startup-and-shutdown-procedure/

Notes:

If you get "TNS-12541: TNS:no listener" when running the first command above, try doing: $ORACLE_HOME/bin/lsnrctl start :)
After that, you may need to restart the Vontu services. Then you should be able to login to https://vontuIP/ or https://vontuIP:8443

Here's a quick script to restart the services on the Enforce server. Add VontuMonitor.sh if single-tier:
cd /opt/Vontu/Protect/bin

./VontuUpdate.sh stop
./VontuIncidentPersister.sh stop
./VontuManager.sh stop
./VontuMonitorController.sh stop
./VontuNotifier.sh stop
./VontuNotifier.sh start
./VontuManager.sh start
./VontuIncidentPersister.sh start
./VontuUpdate.sh start
./VontuMonitorController.sh start



Three-tier DLP Installation - The Oracle Client

There is little in the way of documentation for this.

So, I unwrapped Oracle_11.2.0.1.0_64bit_Client_Lin. Start reading the docs, i.e., the Client Quick Installation Guide. Followed all the instructions in there, including creating the oracle account, etc. For user oracle creation, running oracle_prepare.sh from 11g_r2_32_bit_Installation_Tools.tar.gz works.

Then in order to get past the DLP installation wizard's Oracle page, i.e. error "sqlplus is not executable at /opt/oracle/product/11.2.0/db_1/bin", I needed to do the following for the root account:

If you did not install the Oracle client to /opt/oracle/product/11.2.0/db_1/ then perform the following.
Add the following lines to the root .bash_profile:
export ORACLE_BASE=/u01/app/oracle
export ORACLE_HOME=/u01/app/oracle/product/11.2.0/client_1
LD_LIBRARY_PATH=$ORACLE_HOME:/lib:/usr/lib
LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib
export LD_LIBRARY_PATH

PATH=$PATH:$HOME/bin:$ORACLE_HOME

Then,
. ~/.bash_profile to set the environment.
Then,
mkdir $ORACLE_HOME/bin
cd $ORACLE_HOME/bin
ln -s ../sqlplus
mkdir $ORACLE_HOME/lib


Copy all *.so, and so.11.1 files to $ORACLE_HOME/lib
I also created $ORACLE_HOME/jdbc/lib and to it copied ojdbc*.jar from $ORACLE_HOME.

Be sure to:

set the Base directory and Home directory to match the ORACLE_HOME and ORACLE_BASE paths above in the installation wizard of DLP.

SSH Access to PIX

First generate a public key:

ca gen rsa key 1024
or 512 or 768 for modulus size (less secure)

ca save all

ssh 0.0.0.0 0.0.0.0 inside
or outside
ssh timeout 60
passwd mypassword123

Then ssh in with user: pix

ref:
see http://www.ciscopress.com/articles/article.asp?p=24664&seqNum=5
http://www.velocityreviews.com/forums/t57033-ssh-on-pix-506e-login-name.html