Trying to limit access to ssh via OD

I was wondering if someone could enlighten me about wether this is even possible and if it is how can I accomplish it. I set up an OD and for the time being are trying not to run kerberos services. I also have a number of OSX Client machines that are bound to the OD. Using Workgroup Manager under the "Accounts" section I can create a "computer list" that contains any number of OSX client machines and restrict access to those machines based on user groups. This allows me to restrict AFP and console access to the machines in the group, however it seems that ANY user in the OD can still ssh into the machine. Is there a way to fix this so that I can completely control who has access to those machines? Client machines are OSX 10.4.10 Server Machine is OSX Server 10.4.11 I would appreciate any help on this issue. Thanks, -J