I run Linux at home, at work I'm mainly a C guy, I have done a bit of Java, though it was all sockets/networking stuff, never touched the GUI side of it.
That said if you want to start developing at home you'll want a decent IDE (Integrated Developement Environment), if you were using a command like SSH or Telnet (maybe a program called putty) to connect to a unix box you were probably using something like Emacs or VI, to write the program. There are a number of different IDE's out there which make life a lot easier for you.
In the Linux world "Eclipse" is the IDE of choice, it's a free open source development environment, thats used by a lot of large comercial entities, I think its pretty awesome, it has a massive community writing a heap of plug-ins for it, it's also available for the Windows OS, though I've had no experience running it on that platform.
http://www.eclipse.org/ is there homepage, check that site out, again I'm not sure how it works on Windows but in the Linux world its pretty easy to get everything set up as it resolves all your dependencies with syterm libraries and JRE's for you.
There's some instruction on installing Eclipse
http://wiki.eclipse.org/index.php/Eclipse_FAQs and how to use it etc here.
Here's an article on getting swing set up in Eclipse
http://www-128.ibm.com/developerworks/library/os-swing/http://eos.sourceforge.net/ here's an Eclipse Plug-in to write Swing code.
Google is pretty handy if you have some spare time there are a heap of articles on Java programming and Eclipse, good luck.
